2023-07-08T12:25:45.815 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-08T12:25:45.816 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} --owner scheduled_rishabh@teuthology --name rishabh-2023-07-08_12:15:41-fs-wip-rishabh-mdscaps-2-testing-default-smithi --verbose --archive /home/teuthworker/archive/rishabh-2023-07-08_12:15:41-fs-wip-rishabh-mdscaps-2-testing-default-smithi/7329068 -- /home/teuthworker/archive/rishabh-2023-07-08_12:15:41-fs-wip-rishabh-mdscaps-2-testing-default-smithi/7329068/orig.config.yaml 2023-07-08T12:25:45.858 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-08T12:25:45.947 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-08_12:15:41-fs-wip-rishabh-mdscaps-2-testing-default-smithi/7329068 branch: wip-rishabh-mdscaps-2 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} email: null first_in_suite: false job_id: '7329068' kernel: client: sha1: distro flavor: default kdb: true sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-08_12:15:41-fs-wip-rishabh-mdscaps-2-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-mdscaps-2 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: f3a06ccd0ae55e9794bc8ce562926656b2738063 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f3a06ccd0ae55e9794bc8ce562926656b2738063 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-mdscaps-2 sha1: f3a06ccd0ae55e9794bc8ce562926656b2738063 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: f3a06ccd0ae55e9794bc8ce562926656b2738063 sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-mdscaps-2 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: f3a06ccd0ae55e9794bc8ce562926656b2738063 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi160.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b53f236c1077f1d295baafb9e0ca9a4b447019d7 timestamp: 2023-07-08_12:15:41 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-08T12:25:45.947 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa; will attempt to use it 2023-07-08T12:25:45.948 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks 2023-07-08T12:25:45.948 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-08T12:25:45.950 INFO:teuthology.task.internal:Checking packages... 2023-07-08T12:25:45.976 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'f3a06ccd0ae55e9794bc8ce562926656b2738063' 2023-07-08T12:25:45.976 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-08T12:25:45.977 INFO:teuthology.packaging:ref: None 2023-07-08T12:25:45.977 INFO:teuthology.packaging:tag: None 2023-07-08T12:25:45.977 INFO:teuthology.packaging:branch: wip-rishabh-mdscaps-2 2023-07-08T12:25:45.977 INFO:teuthology.packaging:sha1: f3a06ccd0ae55e9794bc8ce562926656b2738063 2023-07-08T12:25:45.978 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-mdscaps-2 2023-07-08T12:25:46.190 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4738.gf3a06ccd 2023-07-08T12:25:46.191 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-08T12:25:46.197 INFO:teuthology.task.internal:no buildpackages task found 2023-07-08T12:25:46.197 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-08T12:25:46.218 INFO:teuthology.task.internal:Saving configuration 2023-07-08T12:25:46.233 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-08T12:25:46.252 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-08T12:25:46.278 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-08_12:15:41-fs-wip-rishabh-mdscaps-2-testing-default-smithi/7329068', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-08 12:16:56.132724', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-08T12:25:46.306 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-08_12:15:41-fs-wip-rishabh-mdscaps-2-testing-default-smithi/7329068', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-08 12:16:56.134596', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-08T12:25:46.306 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-08T12:25:46.313 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-07-08T12:25:46.313 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-07-08T12:25:46.313 INFO:teuthology.run_tasks:Running task console_log... 2023-07-08T12:25:46.405 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7798d4b280>, signals=[15]) 2023-07-08T12:25:46.406 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-08T12:25:46.412 INFO:teuthology.task.internal:Opening connections... 2023-07-08T12:25:46.413 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2023-07-08T12:25:46.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T12:25:46.482 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2023-07-08T12:25:46.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T12:25:46.550 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-08T12:25:46.556 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2023-07-08T12:25:46.578 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2023-07-08T12:25:46.579 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2023-07-08T12:25:46.638 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Red Hat Enterprise Linux" 2023-07-08T12:25:46.638 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="8.6 (Ootpa)" 2023-07-08T12:25:46.639 INFO:teuthology.orchestra.run.smithi012.stdout:ID="rhel" 2023-07-08T12:25:46.639 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="fedora" 2023-07-08T12:25:46.639 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="8.6" 2023-07-08T12:25:46.639 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el8" 2023-07-08T12:25:46.639 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-08T12:25:46.639 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2023-07-08T12:25:46.639 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-08T12:25:46.640 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.redhat.com/" 2023-07-08T12:25:46.640 INFO:teuthology.orchestra.run.smithi012.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-08T12:25:46.640 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-08T12:25:46.640 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:25:46.640 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-08T12:25:46.640 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-08T12:25:46.640 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-08T12:25:46.641 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-08T12:25:46.641 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2023-07-08T12:25:46.663 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2023-07-08T12:25:46.682 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2023-07-08T12:25:46.683 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2023-07-08T12:25:46.742 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="Red Hat Enterprise Linux" 2023-07-08T12:25:46.742 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="8.6 (Ootpa)" 2023-07-08T12:25:46.742 INFO:teuthology.orchestra.run.smithi160.stdout:ID="rhel" 2023-07-08T12:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="fedora" 2023-07-08T12:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="8.6" 2023-07-08T12:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el8" 2023-07-08T12:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-08T12:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2023-07-08T12:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-08T12:25:46.743 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.redhat.com/" 2023-07-08T12:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-08T12:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-08T12:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-08T12:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-08T12:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-08T12:25:46.744 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-08T12:25:46.745 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2023-07-08T12:25:46.763 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-08T12:25:46.772 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-08T12:25:46.794 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-08T12:25:46.794 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2023-07-08T12:25:46.796 DEBUG:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2023-07-08T12:25:46.814 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-08T12:25:46.827 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-08T12:25:46.827 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2023-07-08T12:25:46.856 DEBUG:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2023-07-08T12:25:46.884 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-08T12:25:46.940 INFO:teuthology.run_tasks:Running task kernel... 2023-07-08T12:25:46.954 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'} 2023-07-08T12:25:46.955 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-07-08T12:25:46.955 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-07-08T12:25:46.955 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-08T12:25:46.956 DEBUG:teuthology.orchestra.run.smithi160:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-08T12:25:46.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-08T12:25:46.974 DEBUG:teuthology.orchestra.run.smithi160:> uname -r 2023-07-08T12:25:46.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-08T12:25:46.975 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2023-07-08T12:25:47.032 INFO:teuthology.orchestra.run.smithi160.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-08T12:25:47.033 INFO:teuthology.task.kernel:Running kernel on smithi160: 4.18.0-372.9.1.el8.x86_64 2023-07-08T12:25:47.033 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum install -y kernel 2023-07-08T12:25:47.035 INFO:teuthology.orchestra.run.smithi012.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-08T12:25:47.035 INFO:teuthology.task.kernel:Running kernel on smithi012: 4.18.0-372.9.1.el8.x86_64 2023-07-08T12:25:47.036 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum install -y kernel 2023-07-08T12:25:48.272 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2023-07-08T12:25:48.272 INFO:teuthology.orchestra.run.smithi160.stdout:Unable to read consumer identity 2023-07-08T12:25:48.334 INFO:teuthology.orchestra.run.smithi012.stdout:Updating Subscription Management repositories. 2023-07-08T12:25:48.334 INFO:teuthology.orchestra.run.smithi012.stdout:Unable to read consumer identity 2023-07-08T12:25:52.070 INFO:teuthology.orchestra.run.smithi160.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-08T12:25:52.400 INFO:teuthology.orchestra.run.smithi012.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2023-07-08T12:25:53.745 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 9.4 MB/s | 16 MB 00:01 2023-07-08T12:25:54.017 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 16 MB 00:01 2023-07-08T12:25:57.178 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 591 kB/s | 24 kB 00:00 2023-07-08T12:25:57.667 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 592 kB/s | 24 kB 00:00 2023-07-08T12:25:57.837 INFO:teuthology.orchestra.run.smithi160.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-07-08T12:25:57.885 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2023-07-08T12:25:57.886 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do. 2023-07-08T12:25:57.887 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2023-07-08T12:25:58.007 DEBUG:teuthology.orchestra.run.smithi160:> echo no | sudo yum reinstall kernel || true 2023-07-08T12:25:58.337 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2023-07-08T12:25:58.337 INFO:teuthology.orchestra.run.smithi160.stdout:Unable to read consumer identity 2023-07-08T12:25:58.360 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-07-08T12:25:58.441 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2023-07-08T12:25:58.442 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2023-07-08T12:25:58.442 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2023-07-08T12:25:58.575 DEBUG:teuthology.orchestra.run.smithi012:> echo no | sudo yum reinstall kernel || true 2023-07-08T12:25:58.598 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:01 ago on Sat 08 Jul 2023 12:25:57 PM UTC. 2023-07-08T12:25:58.638 INFO:teuthology.orchestra.run.smithi160.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-08T12:25:58.676 INFO:teuthology.orchestra.run.smithi160.stderr:Error: No packages marked for reinstall. 2023-07-08T12:25:58.723 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum reinstall -y kernel || true 2023-07-08T12:25:58.927 INFO:teuthology.orchestra.run.smithi012.stdout:Updating Subscription Management repositories. 2023-07-08T12:25:58.927 INFO:teuthology.orchestra.run.smithi012.stdout:Unable to read consumer identity 2023-07-08T12:25:59.053 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2023-07-08T12:25:59.053 INFO:teuthology.orchestra.run.smithi160.stdout:Unable to read consumer identity 2023-07-08T12:25:59.201 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:00:02 ago on Sat 08 Jul 2023 12:25:57 PM UTC. 2023-07-08T12:25:59.245 INFO:teuthology.orchestra.run.smithi012.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-08T12:25:59.300 INFO:teuthology.orchestra.run.smithi012.stderr:Error: No packages marked for reinstall. 2023-07-08T12:25:59.306 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:00:02 ago on Sat 08 Jul 2023 12:25:57 PM UTC. 2023-07-08T12:25:59.346 INFO:teuthology.orchestra.run.smithi160.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-08T12:25:59.347 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum reinstall -y kernel || true 2023-07-08T12:25:59.384 INFO:teuthology.orchestra.run.smithi160.stderr:Error: No packages marked for reinstall. 2023-07-08T12:25:59.430 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q kernel | sort -rV | head -n 1 2023-07-08T12:25:59.509 INFO:teuthology.orchestra.run.smithi160.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-07-08T12:25:59.509 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-07-08T12:25:59.510 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-07-08T12:25:59.510 DEBUG:teuthology.parallel:result is None 2023-07-08T12:25:59.696 INFO:teuthology.orchestra.run.smithi012.stdout:Updating Subscription Management repositories. 2023-07-08T12:25:59.697 INFO:teuthology.orchestra.run.smithi012.stdout:Unable to read consumer identity 2023-07-08T12:25:59.964 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:00:02 ago on Sat 08 Jul 2023 12:25:57 PM UTC. 2023-07-08T12:26:00.008 INFO:teuthology.orchestra.run.smithi012.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-08T12:26:00.043 INFO:teuthology.orchestra.run.smithi012.stderr:Error: No packages marked for reinstall. 2023-07-08T12:26:00.092 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q kernel | sort -rV | head -n 1 2023-07-08T12:26:00.174 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-07-08T12:26:00.175 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-07-08T12:26:00.175 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-07-08T12:26:00.175 DEBUG:teuthology.parallel:result is None 2023-07-08T12:26:00.176 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-08T12:26:00.183 INFO:teuthology.task.internal:Creating test directory... 2023-07-08T12:26:00.183 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-08T12:26:00.185 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-08T12:26:00.205 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-08T12:26:00.218 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-08T12:26:00.233 INFO:teuthology.task.internal:Creating archive directory... 2023-07-08T12:26:00.234 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-08T12:26:00.242 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-08T12:26:00.270 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-08T12:26:00.277 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-08T12:26:00.277 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-08T12:26:00.307 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-08T12:26:00.337 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-08T12:26:00.349 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-08T12:26:00.352 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-08T12:26:00.367 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-08T12:26:00.368 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-08T12:26:00.375 INFO:teuthology.task.internal:Configuring sudo... 2023-07-08T12:26:00.376 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-08T12:26:00.393 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-08T12:26:00.422 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-08T12:26:00.430 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-08T12:26:00.431 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-08T12:26:00.461 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-08T12:26:00.479 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-08T12:26:00.562 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-08T12:26:00.618 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-08T12:26:00.664 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-08T12:26:00.703 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:26:00.704 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-08T12:26:00.772 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-08T12:26:00.823 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-08T12:26:00.870 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-08T12:26:00.954 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-08T12:26:01.000 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:26:01.000 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-08T12:26:01.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2023-07-08T12:26:01.073 DEBUG:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2023-07-08T12:26:01.132 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-08T12:26:01.180 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-08T12:26:01.526 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-08T12:26:01.534 INFO:teuthology.task.internal:Starting timer... 2023-07-08T12:26:01.534 INFO:teuthology.run_tasks:Running task pcp... 2023-07-08T12:26:01.565 INFO:teuthology.run_tasks:Running task selinux... 2023-07-08T12:26:01.578 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-07-08T12:26:01.579 DEBUG:teuthology.orchestra.run.smithi012:> sudo service auditd rotate 2023-07-08T12:26:01.647 INFO:teuthology.orchestra.run.smithi012.stdout:Rotating logs: 2023-07-08T12:26:01.649 DEBUG:teuthology.orchestra.run.smithi160:> sudo service auditd rotate 2023-07-08T12:26:01.708 INFO:teuthology.orchestra.run.smithi160.stdout:Rotating logs: 2023-07-08T12:26:01.709 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-08T12:26:01.710 DEBUG:teuthology.orchestra.run.smithi012:> /usr/sbin/getenforce 2023-07-08T12:26:01.737 INFO:teuthology.orchestra.run.smithi012.stdout:Permissive 2023-07-08T12:26:01.737 DEBUG:teuthology.orchestra.run.smithi160:> /usr/sbin/getenforce 2023-07-08T12:26:01.761 INFO:teuthology.orchestra.run.smithi160.stdout:Permissive 2023-07-08T12:26:01.762 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi012.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi160.front.sepia.ceph.com': 'permissive'} 2023-07-08T12:26:01.762 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-08T12:26:01.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-08T12:26:01.804 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-08T12:26:01.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-08T12:26:01.832 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-08T12:26:01.832 DEBUG:teuthology.orchestra.run.smithi012:> sudo /usr/sbin/setenforce permissive 2023-07-08T12:26:01.874 DEBUG:teuthology.orchestra.run.smithi160:> sudo /usr/sbin/setenforce permissive 2023-07-08T12:26:01.901 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-08T12:26:01.911 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-07-08T12:26:01.912 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-08T12:26:01.929 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-08T12:26:01.931 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi160.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-08T12:34:40.546 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi160.front.sepia.ceph.com')] 2023-07-08T12:34:40.547 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T12:34:40.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T12:34:40.618 DEBUG:teuthology.orchestra.run.smithi012:> true 2023-07-08T12:34:40.686 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T12:34:40.687 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-07-08T12:34:40.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T12:34:40.755 DEBUG:teuthology.orchestra.run.smithi160:> true 2023-07-08T12:34:40.825 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-07-08T12:34:40.826 INFO:teuthology.run_tasks:Running task clock... 2023-07-08T12:34:40.836 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-08T12:34:40.837 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-08T12:34:40.838 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-08T12:34:40.840 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-08T12:34:40.840 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-08T12:34:40.873 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-08T12:34:40.889 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-08T12:34:40.897 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-08T12:34:40.912 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-08T12:34:40.961 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: ntpd: command not found 2023-07-08T12:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout:506 Cannot talk to daemon 2023-07-08T12:34:40.982 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: ntpd: command not found 2023-07-08T12:34:40.989 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-08T12:34:40.993 INFO:teuthology.orchestra.run.smithi160.stdout:506 Cannot talk to daemon 2023-07-08T12:34:41.003 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-08T12:34:41.008 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-08T12:34:41.022 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-08T12:34:41.077 INFO:teuthology.orchestra.run.smithi012.stderr:bash: ntpq: command not found 2023-07-08T12:34:41.080 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-08T12:34:41.081 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2023-07-08T12:34:41.081 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-08T12:34:41.081 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-08T12:34:41.081 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-08T12:34:41.081 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-08T12:34:41.099 INFO:teuthology.orchestra.run.smithi160.stderr:bash: ntpq: command not found 2023-07-08T12:34:41.103 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-08T12:34:41.103 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2023-07-08T12:34:41.104 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-08T12:34:41.104 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-08T12:34:41.104 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-08T12:34:41.104 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-08T12:34:41.105 INFO:teuthology.run_tasks:Running task install... 2023-07-08T12:34:41.113 DEBUG:teuthology.task.install:project ceph 2023-07-08T12:34:41.114 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f3a06ccd0ae55e9794bc8ce562926656b2738063'}} 2023-07-08T12:34:41.114 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f3a06ccd0ae55e9794bc8ce562926656b2738063'} 2023-07-08T12:34:41.114 INFO:teuthology.task.install:Using flavor: default 2023-07-08T12:34:41.120 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-08T12:34:41.120 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-08T12:34:41.121 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f3a06ccd0ae55e9794bc8ce562926656b2738063', 'tag': None, 'wait_for_package': False} 2023-07-08T12:34:41.121 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f3a06ccd0ae55e9794bc8ce562926656b2738063 2023-07-08T12:34:41.123 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f3a06ccd0ae55e9794bc8ce562926656b2738063', 'tag': None, 'wait_for_package': False} 2023-07-08T12:34:41.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f3a06ccd0ae55e9794bc8ce562926656b2738063 2023-07-08T12:34:41.269 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-mdscaps-2/f3a06ccd0ae55e9794bc8ce562926656b2738063/centos/8/flavors/default/ 2023-07-08T12:34:41.269 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4738.gf3a06ccd 2023-07-08T12:34:41.275 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-mdscaps-2/f3a06ccd0ae55e9794bc8ce562926656b2738063/centos/8/flavors/default/ 2023-07-08T12:34:41.276 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4738.gf3a06ccd 2023-07-08T12:34:41.389 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-mdscaps-2/f3a06ccd0ae55e9794bc8ce562926656b2738063/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-mdscaps-2/f3a06ccd0ae55e9794bc8ce562926656b2738063/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-mdscaps-2/f3a06ccd0ae55e9794bc8ce562926656b2738063/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-08T12:34:41.390 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:34:41.390 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-08T12:34:41.418 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-08T12:34:41.419 DEBUG:teuthology.orchestra.run.smithi160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f3a06ccd0ae55e9794bc8ce562926656b2738063/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-08T12:34:41.446 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-mdscaps-2/f3a06ccd0ae55e9794bc8ce562926656b2738063/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-mdscaps-2/f3a06ccd0ae55e9794bc8ce562926656b2738063/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-mdscaps-2/f3a06ccd0ae55e9794bc8ce562926656b2738063/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-08T12:34:41.446 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:34:41.446 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-08T12:34:41.477 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-08T12:34:41.477 DEBUG:teuthology.orchestra.run.smithi012:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f3a06ccd0ae55e9794bc8ce562926656b2738063/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-08T12:34:41.487 DEBUG:teuthology.orchestra.run.smithi160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-08T12:34:41.548 DEBUG:teuthology.orchestra.run.smithi012:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-08T12:34:41.565 DEBUG:teuthology.orchestra.run.smithi160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-08T12:34:41.630 DEBUG:teuthology.orchestra.run.smithi012:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-08T12:34:41.631 INFO:teuthology.orchestra.run.smithi160.stdout:check_obsoletes = 1 2023-07-08T12:34:41.632 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum clean all 2023-07-08T12:34:41.698 INFO:teuthology.orchestra.run.smithi012.stdout:check_obsoletes = 1 2023-07-08T12:34:41.699 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum clean all 2023-07-08T12:34:42.005 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2023-07-08T12:34:42.092 INFO:teuthology.orchestra.run.smithi012.stdout:Updating Subscription Management repositories. 2023-07-08T12:34:42.300 INFO:teuthology.orchestra.run.smithi160.stdout:44 files removed 2023-07-08T12:34:42.347 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-08T12:34:42.370 INFO:teuthology.orchestra.run.smithi012.stdout:44 files removed 2023-07-08T12:34:42.417 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-08T12:34:42.680 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2023-07-08T12:34:42.767 INFO:teuthology.orchestra.run.smithi012.stdout:Updating Subscription Management repositories. 2023-07-08T12:34:43.458 INFO:teuthology.orchestra.run.smithi160.stdout:ceph packages for x86_64 230 kB/s | 81 kB 00:00 2023-07-08T12:34:43.532 INFO:teuthology.orchestra.run.smithi012.stdout:ceph packages for x86_64 235 kB/s | 81 kB 00:00 2023-07-08T12:34:43.824 INFO:teuthology.orchestra.run.smithi160.stdout:ceph noarch packages 28 kB/s | 9.4 kB 00:00 2023-07-08T12:34:43.883 INFO:teuthology.orchestra.run.smithi012.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-07-08T12:34:44.117 INFO:teuthology.orchestra.run.smithi160.stdout:ceph source packages 5.5 kB/s | 1.5 kB 00:00 2023-07-08T12:34:44.201 INFO:teuthology.orchestra.run.smithi012.stdout:ceph source packages 5.1 kB/s | 1.5 kB 00:00 2023-07-08T12:34:46.709 INFO:teuthology.orchestra.run.smithi160.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-08T12:34:46.768 INFO:teuthology.orchestra.run.smithi012.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-07-08T12:34:47.431 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-07-08T12:34:50.271 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 4.5 MB/s | 16 MB 00:03 2023-07-08T12:34:50.893 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 406 kB/s | 24 kB 00:00 2023-07-08T12:34:51.803 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 47 MB 00:00 2023-07-08T12:34:53.943 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 382 kB/s | 24 kB 00:00 2023-07-08T12:34:54.945 INFO:teuthology.orchestra.run.smithi012.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 MB/s | 53 MB 00:00 2023-07-08T12:34:59.678 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-07-08T12:35:03.512 INFO:teuthology.orchestra.run.smithi012.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 MB/s | 47 MB 00:00 2023-07-08T12:35:10.881 INFO:teuthology.orchestra.run.smithi160.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-08T12:35:10.882 INFO:teuthology.orchestra.run.smithi160.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-08T12:35:10.896 INFO:teuthology.orchestra.run.smithi160.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-08T12:35:10.897 INFO:teuthology.orchestra.run.smithi160.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-08T12:35:10.898 INFO:teuthology.orchestra.run.smithi160.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-08T12:35:10.899 INFO:teuthology.orchestra.run.smithi160.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-08T12:35:10.902 INFO:teuthology.orchestra.run.smithi160.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-08T12:35:10.905 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-08T12:35:10.906 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-08T12:35:11.037 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2023-07-08T12:35:11.049 INFO:teuthology.orchestra.run.smithi160.stdout:=========================================================================================================================== 2023-07-08T12:35:11.050 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2023-07-08T12:35:11.050 INFO:teuthology.orchestra.run.smithi160.stdout:=========================================================================================================================== 2023-07-08T12:35:11.050 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2023-07-08T12:35:11.051 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-08T12:35:11.051 INFO:teuthology.orchestra.run.smithi160.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-08T12:35:11.051 INFO:teuthology.orchestra.run.smithi160.stdout: ceph x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 6.4 k 2023-07-08T12:35:11.051 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 5.4 M 2023-07-08T12:35:11.051 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 848 k 2023-07-08T12:35:11.051 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 165 k 2023-07-08T12:35:11.051 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 1.5 M 2023-07-08T12:35:11.052 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 129 k 2023-07-08T12:35:11.052 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 1.7 M 2023-07-08T12:35:11.052 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 7.4 M 2023-07-08T12:35:11.052 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 47 k 2023-07-08T12:35:11.053 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 13 M 2023-07-08T12:35:11.053 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 48 M 2023-07-08T12:35:11.053 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 258 k 2023-07-08T12:35:11.053 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 164 k 2023-07-08T12:35:11.054 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 258 k 2023-07-08T12:35:11.054 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 18 k 2023-07-08T12:35:11.054 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-08T12:35:11.054 INFO:teuthology.orchestra.run.smithi160.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-08T12:35:11.055 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-08T12:35:11.055 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 31 k 2023-07-08T12:35:11.055 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2 x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 771 k 2023-07-08T12:35:11.056 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 139 k 2023-07-08T12:35:11.056 INFO:teuthology.orchestra.run.smithi160.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-08T12:35:11.056 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-08T12:35:11.056 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-08T12:35:11.057 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-08T12:35:11.057 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-08T12:35:11.057 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-08T12:35:11.057 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 197 k 2023-07-08T12:35:11.058 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 371 k 2023-07-08T12:35:11.058 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 364 k 2023-07-08T12:35:11.058 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 107 k 2023-07-08T12:35:11.059 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 94 k 2023-07-08T12:35:11.059 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 3.4 M 2023-07-08T12:35:11.059 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 179 k 2023-07-08T12:35:11.059 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-08T12:35:11.059 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrading: 2023-07-08T12:35:11.060 INFO:teuthology.orchestra.run.smithi160.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-08T12:35:11.060 INFO:teuthology.orchestra.run.smithi160.stdout: librados2 x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 3.7 M 2023-07-08T12:35:11.060 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1 x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 3.6 M 2023-07-08T12:35:11.060 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2023-07-08T12:35:11.061 INFO:teuthology.orchestra.run.smithi160.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-08T12:35:11.061 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 22 M 2023-07-08T12:35:11.061 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 23 k 2023-07-08T12:35:11.061 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 2.3 M 2023-07-08T12:35:11.061 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 240 k 2023-07-08T12:35:11.061 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 4.6 M 2023-07-08T12:35:11.062 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 18 M 2023-07-08T12:35:11.062 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 15 k 2023-07-08T12:35:11.062 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 24 k 2023-07-08T12:35:11.062 INFO:teuthology.orchestra.run.smithi160.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-08T12:35:11.062 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-08T12:35:11.062 INFO:teuthology.orchestra.run.smithi160.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-08T12:35:11.062 INFO:teuthology.orchestra.run.smithi160.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-08T12:35:11.062 INFO:teuthology.orchestra.run.smithi160.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-08T12:35:11.063 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 137 k 2023-07-08T12:35:11.063 INFO:teuthology.orchestra.run.smithi160.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-08T12:35:11.063 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-08T12:35:11.063 INFO:teuthology.orchestra.run.smithi160.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-08T12:35:11.063 INFO:teuthology.orchestra.run.smithi160.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-08T12:35:11.063 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-08T12:35:11.063 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-08T12:35:11.064 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 473 k 2023-07-08T12:35:11.064 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-08T12:35:11.064 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2 x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 6.7 M 2023-07-08T12:35:11.064 INFO:teuthology.orchestra.run.smithi160.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-08T12:35:11.064 INFO:teuthology.orchestra.run.smithi160.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-08T12:35:11.064 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-08T12:35:11.064 INFO:teuthology.orchestra.run.smithi160.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-08T12:35:11.065 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-08T12:35:11.065 INFO:teuthology.orchestra.run.smithi160.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-08T12:35:11.065 INFO:teuthology.orchestra.run.smithi160.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-08T12:35:11.065 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-08T12:35:11.065 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-08T12:35:11.065 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-08T12:35:11.065 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-08T12:35:11.065 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-08T12:35:11.066 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-08T12:35:11.066 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-08T12:35:11.066 INFO:teuthology.orchestra.run.smithi160.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-08T12:35:11.066 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-08T12:35:11.066 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 45 k 2023-07-08T12:35:11.067 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 119 k 2023-07-08T12:35:11.067 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-08T12:35:11.067 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-08T12:35:11.067 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-08T12:35:11.068 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-08T12:35:11.068 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-08T12:35:11.068 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-08T12:35:11.069 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-08T12:35:11.069 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-08T12:35:11.069 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-08T12:35:11.069 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-08T12:35:11.070 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-08T12:35:11.070 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-08T12:35:11.070 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-08T12:35:11.071 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-08T12:35:11.071 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-08T12:35:11.071 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-08T12:35:11.071 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-08T12:35:11.072 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-08T12:35:11.072 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-08T12:35:11.072 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-08T12:35:11.073 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-08T12:35:11.073 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-08T12:35:11.073 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-08T12:35:11.073 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-08T12:35:11.074 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-08T12:35:11.074 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-08T12:35:11.074 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-08T12:35:11.074 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-08T12:35:11.074 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-08T12:35:11.074 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-08T12:35:11.074 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-08T12:35:11.075 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-08T12:35:11.075 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-08T12:35:11.075 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-08T12:35:11.075 INFO:teuthology.orchestra.run.smithi160.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-08T12:35:11.075 INFO:teuthology.orchestra.run.smithi160.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-08T12:35:11.075 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-08T12:35:11.075 INFO:teuthology.orchestra.run.smithi160.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-08T12:35:11.075 INFO:teuthology.orchestra.run.smithi160.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-08T12:35:11.076 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-08T12:35:11.076 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-08T12:35:11.076 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-08T12:35:11.076 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-08T12:35:11.076 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-08T12:35:11.076 INFO:teuthology.orchestra.run.smithi160.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-08T12:35:11.076 INFO:teuthology.orchestra.run.smithi160.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-08T12:35:11.077 INFO:teuthology.orchestra.run.smithi160.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-08T12:35:11.077 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-08T12:35:11.077 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-08T12:35:11.077 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling module streams: 2023-07-08T12:35:11.077 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql 10 2023-07-08T12:35:11.077 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:35:11.077 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2023-07-08T12:35:11.077 INFO:teuthology.orchestra.run.smithi160.stdout:=========================================================================================================================== 2023-07-08T12:35:11.078 INFO:teuthology.orchestra.run.smithi160.stdout:Install 128 Packages 2023-07-08T12:35:11.078 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrade 3 Packages 2023-07-08T12:35:11.078 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:35:11.078 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 205 M 2023-07-08T12:35:11.079 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2023-07-08T12:35:11.587 INFO:teuthology.orchestra.run.smithi160.stdout:(1/131): ceph-18.0.0-4738.gf3a06ccd.el8.x86_64. 46 kB/s | 6.4 kB 00:00 2023-07-08T12:35:11.804 INFO:teuthology.orchestra.run.smithi160.stdout:(2/131): ceph-fuse-18.0.0-4738.gf3a06ccd.el8.x8 3.8 MB/s | 848 kB 00:00 2023-07-08T12:35:11.871 INFO:teuthology.orchestra.run.smithi160.stdout:(3/131): ceph-immutable-object-cache-18.0.0-473 2.4 MB/s | 165 kB 00:00 2023-07-08T12:35:12.155 INFO:teuthology.orchestra.run.smithi160.stdout:(4/131): ceph-base-18.0.0-4738.gf3a06ccd.el8.x8 7.6 MB/s | 5.4 MB 00:00 2023-07-08T12:35:12.230 INFO:teuthology.orchestra.run.smithi160.stdout:(5/131): ceph-mds-18.0.0-4738.gf3a06ccd.el8.x86 6.4 MB/s | 2.3 MB 00:00 2023-07-08T12:35:12.346 INFO:teuthology.orchestra.run.smithi160.stdout:(6/131): ceph-mgr-18.0.0-4738.gf3a06ccd.el8.x86 8.1 MB/s | 1.5 MB 00:00 2023-07-08T12:35:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:(7/131): ceph-mon-18.0.0-4738.gf3a06ccd.el8.x86 6.1 MB/s | 4.6 MB 00:00 2023-07-08T12:35:13.564 INFO:teuthology.orchestra.run.smithi160.stdout:(8/131): ceph-common-18.0.0-4738.gf3a06ccd.el8. 10 MB/s | 22 MB 00:02 2023-07-08T12:35:13.656 INFO:teuthology.orchestra.run.smithi160.stdout:(9/131): ceph-selinux-18.0.0-4738.gf3a06ccd.el8 266 kB/s | 24 kB 00:00 2023-07-08T12:35:14.206 INFO:teuthology.orchestra.run.smithi160.stdout:(10/131): ceph-radosgw-18.0.0-4738.gf3a06ccd.el 11 MB/s | 13 MB 00:01 2023-07-08T12:35:14.456 INFO:teuthology.orchestra.run.smithi160.stdout:(11/131): ceph-osd-18.0.0-4738.gf3a06ccd.el8.x8 8.4 MB/s | 18 MB 00:02 2023-07-08T12:35:14.474 INFO:teuthology.orchestra.run.smithi160.stdout:(12/131): cephfs-mirror-18.0.0-4738.gf3a06ccd.e 967 kB/s | 258 kB 00:00 2023-07-08T12:35:14.515 INFO:teuthology.orchestra.run.smithi160.stdout:(13/131): libcephfs-devel-18.0.0-4738.gf3a06ccd 530 kB/s | 31 kB 00:00 2023-07-08T12:35:14.557 INFO:teuthology.orchestra.run.smithi160.stdout:(14/131): libcephfs2-18.0.0-4738.gf3a06ccd.el8. 9.0 MB/s | 771 kB 00:00 2023-07-08T12:35:14.582 INFO:teuthology.orchestra.run.smithi160.stdout:(15/131): libcephsqlite-18.0.0-4738.gf3a06ccd.e 2.0 MB/s | 137 kB 00:00 2023-07-08T12:35:14.616 INFO:teuthology.orchestra.run.smithi160.stdout:(16/131): librados-devel-18.0.0-4738.gf3a06ccd. 2.3 MB/s | 139 kB 00:00 2023-07-08T12:35:14.675 INFO:teuthology.orchestra.run.smithi160.stdout:(17/131): libradosstriper1-18.0.0-4738.gf3a06cc 5.0 MB/s | 473 kB 00:00 2023-07-08T12:35:14.750 INFO:teuthology.orchestra.run.smithi160.stdout:(18/131): python3-ceph-argparse-18.0.0-4738.gf3 597 kB/s | 45 kB 00:00 2023-07-08T12:35:14.817 INFO:teuthology.orchestra.run.smithi160.stdout:(19/131): python3-ceph-common-18.0.0-4738.gf3a0 1.7 MB/s | 119 kB 00:00 2023-07-08T12:35:14.892 INFO:teuthology.orchestra.run.smithi160.stdout:(20/131): python3-cephfs-18.0.0-4738.gf3a06ccd. 2.6 MB/s | 197 kB 00:00 2023-07-08T12:35:14.984 INFO:teuthology.orchestra.run.smithi160.stdout:(21/131): python3-rados-18.0.0-4738.gf3a06ccd.e 4.0 MB/s | 371 kB 00:00 2023-07-08T12:35:15.093 INFO:teuthology.orchestra.run.smithi160.stdout:(22/131): python3-rbd-18.0.0-4738.gf3a06ccd.el8 3.3 MB/s | 364 kB 00:00 2023-07-08T12:35:15.119 INFO:teuthology.orchestra.run.smithi012.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-08T12:35:15.120 INFO:teuthology.orchestra.run.smithi012.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-08T12:35:15.135 INFO:teuthology.orchestra.run.smithi012.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-08T12:35:15.136 INFO:teuthology.orchestra.run.smithi012.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-08T12:35:15.137 INFO:teuthology.orchestra.run.smithi012.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-08T12:35:15.139 INFO:teuthology.orchestra.run.smithi012.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-08T12:35:15.141 INFO:teuthology.orchestra.run.smithi012.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-08T12:35:15.145 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-08T12:35:15.146 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-08T12:35:15.176 INFO:teuthology.orchestra.run.smithi160.stdout:(23/131): librgw2-18.0.0-4738.gf3a06ccd.el8.x86 12 MB/s | 6.7 MB 00:00 2023-07-08T12:35:15.201 INFO:teuthology.orchestra.run.smithi160.stdout:(24/131): python3-rgw-18.0.0-4738.gf3a06ccd.el8 981 kB/s | 107 kB 00:00 2023-07-08T12:35:15.227 INFO:teuthology.orchestra.run.smithi160.stdout:(25/131): rbd-fuse-18.0.0-4738.gf3a06ccd.el8.x8 1.8 MB/s | 94 kB 00:00 2023-07-08T12:35:15.288 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2023-07-08T12:35:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:(26/131): rbd-nbd-18.0.0-4738.gf3a06ccd.el8.x86 2.6 MB/s | 179 kB 00:00 2023-07-08T12:35:15.301 INFO:teuthology.orchestra.run.smithi012.stdout:=========================================================================================================================== 2023-07-08T12:35:15.301 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2023-07-08T12:35:15.302 INFO:teuthology.orchestra.run.smithi012.stdout:=========================================================================================================================== 2023-07-08T12:35:15.302 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2023-07-08T12:35:15.302 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-08T12:35:15.302 INFO:teuthology.orchestra.run.smithi012.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-08T12:35:15.302 INFO:teuthology.orchestra.run.smithi012.stdout: ceph x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 6.4 k 2023-07-08T12:35:15.302 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 5.4 M 2023-07-08T12:35:15.302 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 848 k 2023-07-08T12:35:15.303 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 165 k 2023-07-08T12:35:15.303 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 1.5 M 2023-07-08T12:35:15.303 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 129 k 2023-07-08T12:35:15.303 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 1.7 M 2023-07-08T12:35:15.303 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 7.4 M 2023-07-08T12:35:15.303 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 47 k 2023-07-08T12:35:15.303 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 13 M 2023-07-08T12:35:15.304 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 48 M 2023-07-08T12:35:15.304 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 258 k 2023-07-08T12:35:15.304 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 164 k 2023-07-08T12:35:15.304 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 258 k 2023-07-08T12:35:15.304 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 18 k 2023-07-08T12:35:15.304 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-08T12:35:15.304 INFO:teuthology.orchestra.run.smithi012.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-08T12:35:15.305 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-08T12:35:15.305 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 31 k 2023-07-08T12:35:15.305 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2 x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 771 k 2023-07-08T12:35:15.305 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 139 k 2023-07-08T12:35:15.305 INFO:teuthology.orchestra.run.smithi012.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-08T12:35:15.305 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-08T12:35:15.305 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-08T12:35:15.306 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-08T12:35:15.306 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-08T12:35:15.306 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-08T12:35:15.306 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 197 k 2023-07-08T12:35:15.306 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 371 k 2023-07-08T12:35:15.306 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 364 k 2023-07-08T12:35:15.306 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 107 k 2023-07-08T12:35:15.306 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 94 k 2023-07-08T12:35:15.307 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 3.4 M 2023-07-08T12:35:15.307 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 179 k 2023-07-08T12:35:15.307 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-08T12:35:15.307 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrading: 2023-07-08T12:35:15.307 INFO:teuthology.orchestra.run.smithi012.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-08T12:35:15.307 INFO:teuthology.orchestra.run.smithi012.stdout: librados2 x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 3.7 M 2023-07-08T12:35:15.308 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1 x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 3.6 M 2023-07-08T12:35:15.308 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2023-07-08T12:35:15.308 INFO:teuthology.orchestra.run.smithi012.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-08T12:35:15.308 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 22 M 2023-07-08T12:35:15.308 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 23 k 2023-07-08T12:35:15.308 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 2.3 M 2023-07-08T12:35:15.308 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 240 k 2023-07-08T12:35:15.309 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 4.6 M 2023-07-08T12:35:15.309 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 18 M 2023-07-08T12:35:15.309 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4738.gf3a06ccd.el8 ceph-noarch 15 k 2023-07-08T12:35:15.309 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 24 k 2023-07-08T12:35:15.309 INFO:teuthology.orchestra.run.smithi012.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-08T12:35:15.309 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-08T12:35:15.309 INFO:teuthology.orchestra.run.smithi012.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-08T12:35:15.310 INFO:teuthology.orchestra.run.smithi012.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-08T12:35:15.310 INFO:teuthology.orchestra.run.smithi012.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-08T12:35:15.310 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 137 k 2023-07-08T12:35:15.310 INFO:teuthology.orchestra.run.smithi012.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-08T12:35:15.310 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-08T12:35:15.310 INFO:teuthology.orchestra.run.smithi012.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-08T12:35:15.310 INFO:teuthology.orchestra.run.smithi012.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-08T12:35:15.310 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-08T12:35:15.311 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-08T12:35:15.311 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1 x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 473 k 2023-07-08T12:35:15.311 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-08T12:35:15.311 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2 x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 6.7 M 2023-07-08T12:35:15.311 INFO:teuthology.orchestra.run.smithi012.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-08T12:35:15.311 INFO:teuthology.orchestra.run.smithi012.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-08T12:35:15.311 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-08T12:35:15.312 INFO:teuthology.orchestra.run.smithi012.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-08T12:35:15.312 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-08T12:35:15.312 INFO:teuthology.orchestra.run.smithi012.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-08T12:35:15.312 INFO:teuthology.orchestra.run.smithi012.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-08T12:35:15.312 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-08T12:35:15.312 INFO:teuthology.orchestra.run.smithi012.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-08T12:35:15.312 INFO:teuthology.orchestra.run.smithi012.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-08T12:35:15.312 INFO:teuthology.orchestra.run.smithi012.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-08T12:35:15.313 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-08T12:35:15.313 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-08T12:35:15.313 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-08T12:35:15.313 INFO:teuthology.orchestra.run.smithi012.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-08T12:35:15.313 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-08T12:35:15.313 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 45 k 2023-07-08T12:35:15.313 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common x86_64 2:18.0.0-4738.gf3a06ccd.el8 ceph 119 k 2023-07-08T12:35:15.314 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-08T12:35:15.314 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-08T12:35:15.314 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-08T12:35:15.314 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-08T12:35:15.314 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-08T12:35:15.314 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-08T12:35:15.314 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-08T12:35:15.314 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-08T12:35:15.315 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-08T12:35:15.315 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-08T12:35:15.315 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-08T12:35:15.315 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-08T12:35:15.315 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-08T12:35:15.315 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-08T12:35:15.315 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-08T12:35:15.316 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-08T12:35:15.316 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-08T12:35:15.316 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-08T12:35:15.316 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-08T12:35:15.316 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-08T12:35:15.316 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-08T12:35:15.317 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-08T12:35:15.317 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-08T12:35:15.317 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-08T12:35:15.318 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-08T12:35:15.318 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-08T12:35:15.318 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-08T12:35:15.318 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-08T12:35:15.318 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-08T12:35:15.318 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-08T12:35:15.318 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-08T12:35:15.318 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-08T12:35:15.319 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-08T12:35:15.319 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-08T12:35:15.319 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-08T12:35:15.319 INFO:teuthology.orchestra.run.smithi012.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-08T12:35:15.319 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-08T12:35:15.319 INFO:teuthology.orchestra.run.smithi012.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-08T12:35:15.319 INFO:teuthology.orchestra.run.smithi012.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-08T12:35:15.320 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-08T12:35:15.320 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-08T12:35:15.320 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-08T12:35:15.320 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-08T12:35:15.320 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-08T12:35:15.320 INFO:teuthology.orchestra.run.smithi012.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-08T12:35:15.320 INFO:teuthology.orchestra.run.smithi012.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-08T12:35:15.321 INFO:teuthology.orchestra.run.smithi012.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-08T12:35:15.321 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-08T12:35:15.321 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-08T12:35:15.321 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling module streams: 2023-07-08T12:35:15.321 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql 10 2023-07-08T12:35:15.321 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:35:15.321 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2023-07-08T12:35:15.321 INFO:teuthology.orchestra.run.smithi012.stdout:=========================================================================================================================== 2023-07-08T12:35:15.322 INFO:teuthology.orchestra.run.smithi012.stdout:Install 128 Packages 2023-07-08T12:35:15.322 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrade 3 Packages 2023-07-08T12:35:15.322 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:35:15.322 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 205 M 2023-07-08T12:35:15.323 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2023-07-08T12:35:15.352 INFO:teuthology.orchestra.run.smithi160.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4738.g 396 kB/s | 23 kB 00:00 2023-07-08T12:35:15.419 INFO:teuthology.orchestra.run.smithi160.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4738.gf3a06cc 1.9 MB/s | 129 kB 00:00 2023-07-08T12:35:15.528 INFO:teuthology.orchestra.run.smithi012.stdout:(1/131): ceph-18.0.0-4738.gf3a06ccd.el8.x86_64. 36 kB/s | 6.4 kB 00:00 2023-07-08T12:35:15.636 INFO:teuthology.orchestra.run.smithi160.stdout:(29/131): rbd-mirror-18.0.0-4738.gf3a06ccd.el8. 7.9 MB/s | 3.4 MB 00:00 2023-07-08T12:35:15.678 INFO:teuthology.orchestra.run.smithi160.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4738.gf3a06 6.6 MB/s | 1.7 MB 00:00 2023-07-08T12:35:15.770 INFO:teuthology.orchestra.run.smithi160.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4738.gf3 2.6 MB/s | 240 kB 00:00 2023-07-08T12:35:15.864 INFO:teuthology.orchestra.run.smithi160.stdout:(32/131): ceph-mgr-rook-18.0.0-4738.gf3a06ccd.e 517 kB/s | 47 kB 00:00 2023-07-08T12:35:15.928 INFO:teuthology.orchestra.run.smithi012.stdout:(2/131): ceph-fuse-18.0.0-4738.gf3a06ccd.el8.x8 2.1 MB/s | 848 kB 00:00 2023-07-08T12:35:15.996 INFO:teuthology.orchestra.run.smithi160.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4738.gf 109 kB/s | 15 kB 00:00 2023-07-08T12:35:16.012 INFO:teuthology.orchestra.run.smithi012.stdout:(3/131): ceph-immutable-object-cache-18.0.0-473 1.9 MB/s | 165 kB 00:00 2023-07-08T12:35:16.088 INFO:teuthology.orchestra.run.smithi160.stdout:(34/131): ceph-volume-18.0.0-4738.gf3a06ccd.el8 2.8 MB/s | 258 kB 00:00 2023-07-08T12:35:16.172 INFO:teuthology.orchestra.run.smithi160.stdout:(35/131): cephadm-18.0.0-4738.gf3a06ccd.el8.noa 1.9 MB/s | 164 kB 00:00 2023-07-08T12:35:16.263 INFO:teuthology.orchestra.run.smithi160.stdout:(36/131): cephfs-top-18.0.0-4738.gf3a06ccd.el8. 196 kB/s | 18 kB 00:00 2023-07-08T12:35:16.722 INFO:teuthology.orchestra.run.smithi160.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 6.8 MB/s | 7.4 MB 00:01 2023-07-08T12:35:16.762 INFO:teuthology.orchestra.run.smithi012.stdout:(4/131): ceph-mds-18.0.0-4738.gf3a06ccd.el8.x86 3.0 MB/s | 2.3 MB 00:00 2023-07-08T12:35:16.831 INFO:teuthology.orchestra.run.smithi160.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 540 kB/s | 306 kB 00:00 2023-07-08T12:35:16.981 INFO:teuthology.orchestra.run.smithi160.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 501 kB/s | 75 kB 00:00 2023-07-08T12:35:17.006 INFO:teuthology.orchestra.run.smithi160.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 207 kB/s | 59 kB 00:00 2023-07-08T12:35:17.074 INFO:teuthology.orchestra.run.smithi160.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 652 kB/s | 44 kB 00:00 2023-07-08T12:35:17.137 INFO:teuthology.orchestra.run.smithi012.stdout:(5/131): ceph-mgr-18.0.0-4738.gf3a06ccd.el8.x86 4.1 MB/s | 1.5 MB 00:00 2023-07-08T12:35:17.174 INFO:teuthology.orchestra.run.smithi160.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-07-08T12:35:17.240 INFO:teuthology.orchestra.run.smithi160.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 494 kB/s | 33 kB 00:00 2023-07-08T12:35:17.308 INFO:teuthology.orchestra.run.smithi160.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 244 kB/s | 16 kB 00:00 2023-07-08T12:35:17.399 INFO:teuthology.orchestra.run.smithi160.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2023-07-08T12:35:17.508 INFO:teuthology.orchestra.run.smithi160.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-07-08T12:35:17.575 INFO:teuthology.orchestra.run.smithi160.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2023-07-08T12:35:17.609 INFO:teuthology.orchestra.run.smithi160.stdout:(48/131): python3-asyncssh-2.7.0-2.el8.noarch.r 759 kB/s | 476 kB 00:00 2023-07-08T12:35:17.642 INFO:teuthology.orchestra.run.smithi160.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-07-08T12:35:17.676 INFO:teuthology.orchestra.run.smithi160.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 262 kB/s | 18 kB 00:00 2023-07-08T12:35:17.785 INFO:teuthology.orchestra.run.smithi160.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 450 kB/s | 49 kB 00:00 2023-07-08T12:35:17.860 INFO:teuthology.orchestra.run.smithi160.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 5.4 MB/s | 1.2 MB 00:00 2023-07-08T12:35:17.895 INFO:teuthology.orchestra.run.smithi160.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 541 kB/s | 59 kB 00:00 2023-07-08T12:35:17.920 INFO:teuthology.orchestra.run.smithi012.stdout:(6/131): ceph-common-18.0.0-4738.gf3a06ccd.el8. 8.6 MB/s | 22 MB 00:02 2023-07-08T12:35:17.927 INFO:teuthology.orchestra.run.smithi160.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 900 kB/s | 60 kB 00:00 2023-07-08T12:35:17.994 INFO:teuthology.orchestra.run.smithi160.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 234 kB/s | 16 kB 00:00 2023-07-08T12:35:18.061 INFO:teuthology.orchestra.run.smithi160.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 504 kB/s | 34 kB 00:00 2023-07-08T12:35:18.096 INFO:teuthology.orchestra.run.smithi012.stdout:(7/131): ceph-mon-18.0.0-4738.gf3a06ccd.el8.x86 4.8 MB/s | 4.6 MB 00:00 2023-07-08T12:35:18.153 INFO:teuthology.orchestra.run.smithi160.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2023-07-08T12:35:18.388 INFO:teuthology.orchestra.run.smithi012.stdout:(8/131): ceph-base-18.0.0-4738.gf3a06ccd.el8.x8 1.8 MB/s | 5.4 MB 00:03 2023-07-08T12:35:18.471 INFO:teuthology.orchestra.run.smithi012.stdout:(9/131): ceph-selinux-18.0.0-4738.gf3a06ccd.el8 293 kB/s | 24 kB 00:00 2023-07-08T12:35:18.595 INFO:teuthology.orchestra.run.smithi160.stdout:(58/131): ceph-test-18.0.0-4738.gf3a06ccd.el8.x 9.7 MB/s | 48 MB 00:04 2023-07-08T12:35:18.620 INFO:teuthology.orchestra.run.smithi160.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 132 kB/s | 61 kB 00:00 2023-07-08T12:35:18.645 INFO:teuthology.orchestra.run.smithi160.stdout:(60/131): python3-pecan-1.3.2-9.el8.noarch.rpm 376 kB/s | 283 kB 00:00 2023-07-08T12:35:18.679 INFO:teuthology.orchestra.run.smithi160.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 408 kB/s | 24 kB 00:00 2023-07-08T12:35:18.712 INFO:teuthology.orchestra.run.smithi160.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 468 kB/s | 29 kB 00:00 2023-07-08T12:35:18.738 INFO:teuthology.orchestra.run.smithi160.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 471 kB/s | 27 kB 00:00 2023-07-08T12:35:18.805 INFO:teuthology.orchestra.run.smithi160.stdout:(64/131): python3-simplegeneric-0.8.1-17.el8.no 89 kB/s | 19 kB 00:00 2023-07-08T12:35:18.830 INFO:teuthology.orchestra.run.smithi160.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.1 MB/s | 255 kB 00:00 2023-07-08T12:35:18.922 INFO:teuthology.orchestra.run.smithi160.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 931 kB/s | 85 kB 00:00 2023-07-08T12:35:18.939 INFO:teuthology.orchestra.run.smithi160.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 457 kB/s | 61 kB 00:00 2023-07-08T12:35:18.981 INFO:teuthology.orchestra.run.smithi160.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 391 kB/s | 23 kB 00:00 2023-07-08T12:35:19.081 INFO:teuthology.orchestra.run.smithi160.stdout:(69/131): python3-webob-1.8.5-1.el8.1.noarch.rp 732 kB/s | 251 kB 00:00 2023-07-08T12:35:19.106 INFO:teuthology.orchestra.run.smithi160.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 557 kB/s | 70 kB 00:00 2023-07-08T12:35:19.113 INFO:teuthology.orchestra.run.smithi012.stdout:(10/131): ceph-osd-18.0.0-4738.gf3a06ccd.el8.x8 15 MB/s | 18 MB 00:01 2023-07-08T12:35:19.172 INFO:teuthology.orchestra.run.smithi012.stdout:(11/131): cephfs-mirror-18.0.0-4738.gf3a06ccd.e 4.3 MB/s | 258 kB 00:00 2023-07-08T12:35:19.182 INFO:teuthology.orchestra.run.smithi160.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-07-08T12:35:19.222 INFO:teuthology.orchestra.run.smithi012.stdout:(12/131): libcephfs-devel-18.0.0-4738.gf3a06ccd 623 kB/s | 31 kB 00:00 2023-07-08T12:35:19.223 INFO:teuthology.orchestra.run.smithi160.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-08T12:35:19.290 INFO:teuthology.orchestra.run.smithi160.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2023-07-08T12:35:19.314 INFO:teuthology.orchestra.run.smithi012.stdout:(13/131): libcephfs2-18.0.0-4738.gf3a06ccd.el8. 8.2 MB/s | 771 kB 00:00 2023-07-08T12:35:19.349 INFO:teuthology.orchestra.run.smithi160.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-07-08T12:35:19.373 INFO:teuthology.orchestra.run.smithi012.stdout:(14/131): libcephsqlite-18.0.0-4738.gf3a06ccd.e 2.3 MB/s | 137 kB 00:00 2023-07-08T12:35:19.399 INFO:teuthology.orchestra.run.smithi160.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 778 kB/s | 39 kB 00:00 2023-07-08T12:35:19.414 INFO:teuthology.orchestra.run.smithi012.stdout:(15/131): librados-devel-18.0.0-4738.gf3a06ccd. 3.3 MB/s | 139 kB 00:00 2023-07-08T12:35:19.498 INFO:teuthology.orchestra.run.smithi012.stdout:(16/131): libradosstriper1-18.0.0-4738.gf3a06cc 5.6 MB/s | 473 kB 00:00 2023-07-08T12:35:19.550 INFO:teuthology.orchestra.run.smithi160.stdout:(76/131): python3-scipy-1.0.0-21.module+el8.5.0 29 MB/s | 14 MB 00:00 2023-07-08T12:35:19.583 INFO:teuthology.orchestra.run.smithi160.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 3.7 MB/s | 688 kB 00:00 2023-07-08T12:35:19.625 INFO:teuthology.orchestra.run.smithi160.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.0 MB/s | 538 kB 00:00 2023-07-08T12:35:19.651 INFO:teuthology.orchestra.run.smithi160.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-07-08T12:35:19.709 INFO:teuthology.orchestra.run.smithi160.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2023-07-08T12:35:19.743 INFO:teuthology.orchestra.run.smithi160.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 7.6 MB/s | 713 kB 00:00 2023-07-08T12:35:19.801 INFO:teuthology.orchestra.run.smithi160.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 455 kB/s | 27 kB 00:00 2023-07-08T12:35:19.877 INFO:teuthology.orchestra.run.smithi160.stdout:(83/131): python3-babel-2.5.1-7.el8.noarch.rpm 29 MB/s | 4.8 MB 00:00 2023-07-08T12:35:19.902 INFO:teuthology.orchestra.run.smithi160.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 3.1 MB/s | 320 kB 00:00 2023-07-08T12:35:19.923 INFO:teuthology.orchestra.run.smithi012.stdout:(17/131): ceph-radosgw-18.0.0-4738.gf3a06ccd.el 7.4 MB/s | 13 MB 00:01 2023-07-08T12:35:19.927 INFO:teuthology.orchestra.run.smithi160.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 876 kB/s | 44 kB 00:00 2023-07-08T12:35:19.994 INFO:teuthology.orchestra.run.smithi160.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 6.7 MB/s | 457 kB 00:00 2023-07-08T12:35:20.024 INFO:teuthology.orchestra.run.smithi012.stdout:(18/131): librgw2-18.0.0-4738.gf3a06ccd.el8.x86 13 MB/s | 6.7 MB 00:00 2023-07-08T12:35:20.049 INFO:teuthology.orchestra.run.smithi012.stdout:(19/131): python3-ceph-argparse-18.0.0-4738.gf3 358 kB/s | 45 kB 00:00 2023-07-08T12:35:20.061 INFO:teuthology.orchestra.run.smithi160.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 24 MB/s | 3.7 MB 00:00 2023-07-08T12:35:20.074 INFO:teuthology.orchestra.run.smithi012.stdout:(20/131): python3-ceph-common-18.0.0-4738.gf3a0 2.3 MB/s | 119 kB 00:00 2023-07-08T12:35:20.087 INFO:teuthology.orchestra.run.smithi160.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.4 MB/s | 225 kB 00:00 2023-07-08T12:35:20.099 INFO:teuthology.orchestra.run.smithi012.stdout:(21/131): python3-cephfs-18.0.0-4738.gf3a06ccd. 3.8 MB/s | 197 kB 00:00 2023-07-08T12:35:20.120 INFO:teuthology.orchestra.run.smithi160.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-07-08T12:35:20.125 INFO:teuthology.orchestra.run.smithi012.stdout:(22/131): python3-rados-18.0.0-4738.gf3a06ccd.e 7.2 MB/s | 371 kB 00:00 2023-07-08T12:35:20.145 INFO:teuthology.orchestra.run.smithi160.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.1 MB/s | 126 kB 00:00 2023-07-08T12:35:20.159 INFO:teuthology.orchestra.run.smithi012.stdout:(23/131): python3-rbd-18.0.0-4738.gf3a06ccd.el8 6.1 MB/s | 364 kB 00:00 2023-07-08T12:35:20.170 INFO:teuthology.orchestra.run.smithi160.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-07-08T12:35:20.183 INFO:teuthology.orchestra.run.smithi012.stdout:(24/131): python3-rgw-18.0.0-4738.gf3a06ccd.el8 1.8 MB/s | 107 kB 00:00 2023-07-08T12:35:20.201 INFO:teuthology.orchestra.run.smithi012.stdout:(25/131): rbd-fuse-18.0.0-4738.gf3a06ccd.el8.x8 2.2 MB/s | 94 kB 00:00 2023-07-08T12:35:20.204 INFO:teuthology.orchestra.run.smithi160.stdout:(92/131): thrift-0.13.0-2.el8.x86_64.rpm 1.4 MB/s | 1.7 MB 00:01 2023-07-08T12:35:20.221 INFO:teuthology.orchestra.run.smithi160.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.3 MB/s | 103 kB 00:00 2023-07-08T12:35:20.263 INFO:teuthology.orchestra.run.smithi160.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 7.5 MB/s | 709 kB 00:00 2023-07-08T12:35:20.287 INFO:teuthology.orchestra.run.smithi012.stdout:(26/131): rbd-nbd-18.0.0-4738.gf3a06ccd.el8.x86 2.0 MB/s | 179 kB 00:00 2023-07-08T12:35:20.288 INFO:teuthology.orchestra.run.smithi160.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 3.1 MB/s | 198 kB 00:00 2023-07-08T12:35:20.351 INFO:teuthology.orchestra.run.smithi012.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4738.g 365 kB/s | 23 kB 00:00 2023-07-08T12:35:20.355 INFO:teuthology.orchestra.run.smithi160.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.0 MB/s | 202 kB 00:00 2023-07-08T12:35:20.410 INFO:teuthology.orchestra.run.smithi012.stdout:(28/131): rbd-mirror-18.0.0-4738.gf3a06ccd.el8. 15 MB/s | 3.4 MB 00:00 2023-07-08T12:35:20.444 INFO:teuthology.orchestra.run.smithi012.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4738.gf3a06cc 1.4 MB/s | 129 kB 00:00 2023-07-08T12:35:20.467 INFO:teuthology.orchestra.run.smithi160.stdout:(97/131): openblas-threads-0.3.15-3.el8.x86_64. 18 MB/s | 4.9 MB 00:00 2023-07-08T12:35:20.498 INFO:teuthology.orchestra.run.smithi160.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.4 MB/s | 354 kB 00:00 2023-07-08T12:35:20.552 INFO:teuthology.orchestra.run.smithi012.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4738.gf3a06 12 MB/s | 1.7 MB 00:00 2023-07-08T12:35:20.581 INFO:teuthology.orchestra.run.smithi160.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-07-08T12:35:20.611 INFO:teuthology.orchestra.run.smithi012.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4738.gf3 4.0 MB/s | 240 kB 00:00 2023-07-08T12:35:20.623 INFO:teuthology.orchestra.run.smithi160.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-07-08T12:35:20.674 INFO:teuthology.orchestra.run.smithi160.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-07-08T12:35:20.690 INFO:teuthology.orchestra.run.smithi012.stdout:(32/131): ceph-mgr-rook-18.0.0-4738.gf3a06ccd.e 600 kB/s | 47 kB 00:00 2023-07-08T12:35:20.691 INFO:teuthology.orchestra.run.smithi160.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 699 kB/s | 47 kB 00:00 2023-07-08T12:35:20.724 INFO:teuthology.orchestra.run.smithi160.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-07-08T12:35:20.741 INFO:teuthology.orchestra.run.smithi160.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 846 kB/s | 43 kB 00:00 2023-07-08T12:35:20.753 INFO:teuthology.orchestra.run.smithi012.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4738.gf 233 kB/s | 15 kB 00:00 2023-07-08T12:35:20.808 INFO:teuthology.orchestra.run.smithi160.stdout:(105/131): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2023-07-08T12:35:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:(34/131): ceph-volume-18.0.0-4738.gf3a06ccd.el8 4.3 MB/s | 258 kB 00:00 2023-07-08T12:35:20.825 INFO:teuthology.orchestra.run.smithi160.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 575 kB/s | 48 kB 00:00 2023-07-08T12:35:20.850 INFO:teuthology.orchestra.run.smithi160.stdout:(107/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-07-08T12:35:20.862 INFO:teuthology.orchestra.run.smithi012.stdout:(35/131): cephadm-18.0.0-4738.gf3a06ccd.el8.noa 3.2 MB/s | 164 kB 00:00 2023-07-08T12:35:20.875 INFO:teuthology.orchestra.run.smithi160.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2023-07-08T12:35:20.892 INFO:teuthology.orchestra.run.smithi160.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 638 kB/s | 43 kB 00:00 2023-07-08T12:35:20.918 INFO:teuthology.orchestra.run.smithi160.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 1.6 MB/s | 109 kB 00:00 2023-07-08T12:35:20.921 INFO:teuthology.orchestra.run.smithi012.stdout:(36/131): cephfs-top-18.0.0-4738.gf3a06ccd.el8. 308 kB/s | 18 kB 00:00 2023-07-08T12:35:20.943 INFO:teuthology.orchestra.run.smithi160.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.3 MB/s | 229 kB 00:00 2023-07-08T12:35:20.976 INFO:teuthology.orchestra.run.smithi160.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 6.5 MB/s | 560 kB 00:00 2023-07-08T12:35:21.002 INFO:teuthology.orchestra.run.smithi160.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 223 kB/s | 19 kB 00:00 2023-07-08T12:35:21.138 INFO:teuthology.orchestra.run.smithi012.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-07-08T12:35:21.265 INFO:teuthology.orchestra.run.smithi160.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 481 kB/s | 155 kB 00:00 2023-07-08T12:35:21.294 INFO:teuthology.orchestra.run.smithi160.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 61 kB/s | 18 kB 00:00 2023-07-08T12:35:21.328 INFO:teuthology.orchestra.run.smithi160.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 1.6 MB/s | 559 kB 00:00 2023-07-08T12:35:21.361 INFO:teuthology.orchestra.run.smithi160.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.2 MB/s | 605 kB 00:00 2023-07-08T12:35:21.386 INFO:teuthology.orchestra.run.smithi160.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-07-08T12:35:21.462 INFO:teuthology.orchestra.run.smithi160.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 17 MB/s | 2.3 MB 00:00 2023-07-08T12:35:21.487 INFO:teuthology.orchestra.run.smithi160.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 577 kB/s | 72 kB 00:00 2023-07-08T12:35:21.512 INFO:teuthology.orchestra.run.smithi160.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 660 kB/s | 83 kB 00:00 2023-07-08T12:35:21.538 INFO:teuthology.orchestra.run.smithi160.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 795 kB/s | 60 kB 00:00 2023-07-08T12:35:21.563 INFO:teuthology.orchestra.run.smithi160.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.3 MB/s | 98 kB 00:00 2023-07-08T12:35:21.588 INFO:teuthology.orchestra.run.smithi160.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 763 kB/s | 58 kB 00:00 2023-07-08T12:35:21.638 INFO:teuthology.orchestra.run.smithi160.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 384 kB/s | 39 kB 00:00 2023-07-08T12:35:21.672 INFO:teuthology.orchestra.run.smithi160.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 5.8 MB/s | 644 kB 00:00 2023-07-08T12:35:21.697 INFO:teuthology.orchestra.run.smithi160.stdout:(127/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.5 MB/s | 171 kB 00:00 2023-07-08T12:35:21.723 INFO:teuthology.orchestra.run.smithi160.stdout:(128/131): NetworkManager-initscripts-updown-1. 1.6 MB/s | 138 kB 00:00 2023-07-08T12:35:21.752 INFO:teuthology.orchestra.run.smithi012.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 96 kB/s | 59 kB 00:00 2023-07-08T12:35:21.823 INFO:teuthology.orchestra.run.smithi160.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 7.8 MB/s | 798 kB 00:00 2023-07-08T12:35:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 308 kB/s | 306 kB 00:00 2023-07-08T12:35:22.005 INFO:teuthology.orchestra.run.smithi012.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 297 kB/s | 75 kB 00:00 2023-07-08T12:35:22.048 INFO:teuthology.orchestra.run.smithi160.stdout:(130/131): librados2-18.0.0-4738.gf3a06ccd.el8. 9.7 MB/s | 3.7 MB 00:00 2023-07-08T12:35:22.122 INFO:teuthology.orchestra.run.smithi012.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 374 kB/s | 44 kB 00:00 2023-07-08T12:35:22.149 INFO:teuthology.orchestra.run.smithi160.stdout:(131/131): librbd1-18.0.0-4738.gf3a06ccd.el8.x8 7.9 MB/s | 3.6 MB 00:00 2023-07-08T12:35:22.153 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2023-07-08T12:35:22.154 INFO:teuthology.orchestra.run.smithi160.stdout:Total 18 MB/s | 205 MB 00:11 2023-07-08T12:35:22.431 INFO:teuthology.orchestra.run.smithi012.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 601 kB/s | 185 kB 00:00 2023-07-08T12:35:22.548 INFO:teuthology.orchestra.run.smithi012.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 283 kB/s | 33 kB 00:00 2023-07-08T12:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout:(44/131): python3-asyncssh-2.7.0-2.el8.noarch.r 616 kB/s | 476 kB 00:00 2023-07-08T12:35:22.723 INFO:teuthology.orchestra.run.smithi012.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 93 kB/s | 16 kB 00:00 2023-07-08T12:35:22.982 INFO:teuthology.orchestra.run.smithi012.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 589 kB/s | 173 kB 00:00 2023-07-08T12:35:22.996 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2023-07-08T12:35:23.114 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2023-07-08T12:35:23.115 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2023-07-08T12:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 390 kB/s | 81 kB 00:00 2023-07-08T12:35:23.307 INFO:teuthology.orchestra.run.smithi012.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 659 kB/s | 384 kB 00:00 2023-07-08T12:35:23.324 INFO:teuthology.orchestra.run.smithi012.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-07-08T12:35:23.450 INFO:teuthology.orchestra.run.smithi012.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 141 kB/s | 18 kB 00:00 2023-07-08T12:35:23.558 INFO:teuthology.orchestra.run.smithi012.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 450 kB/s | 49 kB 00:00 2023-07-08T12:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 542 kB/s | 59 kB 00:00 2023-07-08T12:35:23.784 INFO:teuthology.orchestra.run.smithi012.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 516 kB/s | 60 kB 00:00 2023-07-08T12:35:24.001 INFO:teuthology.orchestra.run.smithi012.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2023-07-08T12:35:24.067 INFO:teuthology.orchestra.run.smithi012.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 1.6 MB/s | 1.2 MB 00:00 2023-07-08T12:35:24.110 INFO:teuthology.orchestra.run.smithi012.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 143 kB/s | 16 kB 00:00 2023-07-08T12:35:24.205 INFO:teuthology.orchestra.run.smithi012.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 245 kB/s | 34 kB 00:00 2023-07-08T12:35:24.232 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2023-07-08T12:35:24.235 INFO:teuthology.orchestra.run.smithi012.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2023-07-08T12:35:24.236 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2023-07-08T12:35:24.319 INFO:teuthology.orchestra.run.smithi012.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 543 kB/s | 61 kB 00:00 2023-07-08T12:35:24.344 INFO:teuthology.orchestra.run.smithi012.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-07-08T12:35:24.419 INFO:teuthology.orchestra.run.smithi012.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-07-08T12:35:24.453 INFO:teuthology.orchestra.run.smithi012.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2023-07-08T12:35:24.528 INFO:teuthology.orchestra.run.smithi012.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2023-07-08T12:35:24.592 INFO:teuthology.orchestra.run.smithi012.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.8 MB/s | 255 kB 00:00 2023-07-08T12:35:24.654 INFO:teuthology.orchestra.run.smithi012.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.0 MB/s | 251 kB 00:00 2023-07-08T12:35:24.704 INFO:teuthology.orchestra.run.smithi012.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 546 kB/s | 61 kB 00:00 2023-07-08T12:35:24.771 INFO:teuthology.orchestra.run.smithi012.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 731 kB/s | 85 kB 00:00 2023-07-08T12:35:24.821 INFO:teuthology.orchestra.run.smithi012.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-07-08T12:35:24.847 INFO:teuthology.orchestra.run.smithi012.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-08T12:35:24.914 INFO:teuthology.orchestra.run.smithi012.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 704 kB/s | 47 kB 00:00 2023-07-08T12:35:24.987 INFO:teuthology.orchestra.run.smithi012.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.7 MB/s | 200 kB 00:00 2023-07-08T12:35:25.039 INFO:teuthology.orchestra.run.smithi012.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 823 kB/s | 43 kB 00:00 2023-07-08T12:35:25.090 INFO:teuthology.orchestra.run.smithi012.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-08T12:35:25.131 INFO:teuthology.orchestra.run.smithi012.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 4.8 MB/s | 1.7 MB 00:00 2023-07-08T12:35:25.157 INFO:teuthology.orchestra.run.smithi012.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 721 kB/s | 48 kB 00:00 2023-07-08T12:35:25.224 INFO:teuthology.orchestra.run.smithi012.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 947 kB/s | 87 kB 00:00 2023-07-08T12:35:25.249 INFO:teuthology.orchestra.run.smithi012.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 511 kB/s | 43 kB 00:00 2023-07-08T12:35:25.274 INFO:teuthology.orchestra.run.smithi012.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-07-08T12:35:25.300 INFO:teuthology.orchestra.run.smithi012.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-07-08T12:35:25.333 INFO:teuthology.orchestra.run.smithi012.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-07-08T12:35:25.392 INFO:teuthology.orchestra.run.smithi012.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 203 kB/s | 19 kB 00:00 2023-07-08T12:35:25.409 INFO:teuthology.orchestra.run.smithi012.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.0 MB/s | 155 kB 00:00 2023-07-08T12:35:25.443 INFO:teuthology.orchestra.run.smithi012.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 11 MB/s | 559 kB 00:00 2023-07-08T12:35:25.477 INFO:teuthology.orchestra.run.smithi012.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 266 kB/s | 18 kB 00:00 2023-07-08T12:35:25.510 INFO:teuthology.orchestra.run.smithi012.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 8.8 MB/s | 605 kB 00:00 2023-07-08T12:35:25.552 INFO:teuthology.orchestra.run.smithi012.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-07-08T12:35:25.602 INFO:teuthology.orchestra.run.smithi012.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-07-08T12:35:25.661 INFO:teuthology.orchestra.run.smithi012.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 15 MB/s | 2.3 MB 00:00 2023-07-08T12:35:25.686 INFO:teuthology.orchestra.run.smithi012.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 991 kB/s | 83 kB 00:00 2023-07-08T12:35:25.711 INFO:teuthology.orchestra.run.smithi012.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-07-08T12:35:25.787 INFO:teuthology.orchestra.run.smithi012.stdout:(91/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 769 kB/s | 58 kB 00:00 2023-07-08T12:35:25.812 INFO:teuthology.orchestra.run.smithi012.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 781 kB/s | 98 kB 00:00 2023-07-08T12:35:25.837 INFO:teuthology.orchestra.run.smithi012.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 767 kB/s | 39 kB 00:00 2023-07-08T12:35:25.871 INFO:teuthology.orchestra.run.smithi012.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-07-08T12:35:25.896 INFO:teuthology.orchestra.run.smithi012.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-07-08T12:35:25.921 INFO:teuthology.orchestra.run.smithi012.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-08T12:35:25.963 INFO:teuthology.orchestra.run.smithi012.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-08T12:35:26.014 INFO:teuthology.orchestra.run.smithi012.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-07-08T12:35:26.064 INFO:teuthology.orchestra.run.smithi012.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-07-08T12:35:26.148 INFO:teuthology.orchestra.run.smithi012.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.8 MB/s | 323 kB 00:00 2023-07-08T12:35:26.198 INFO:teuthology.orchestra.run.smithi012.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 779 kB/s | 39 kB 00:00 2023-07-08T12:35:26.381 INFO:teuthology.orchestra.run.smithi012.stdout:(102/131): python3-scipy-1.0.0-21.module+el8.5. 28 MB/s | 14 MB 00:00 2023-07-08T12:35:26.415 INFO:teuthology.orchestra.run.smithi012.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 3.1 MB/s | 688 kB 00:00 2023-07-08T12:35:26.457 INFO:teuthology.orchestra.run.smithi012.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-07-08T12:35:26.509 INFO:teuthology.orchestra.run.smithi012.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.6 MB/s | 157 kB 00:00 2023-07-08T12:35:26.541 INFO:teuthology.orchestra.run.smithi012.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2023-07-08T12:35:26.574 INFO:teuthology.orchestra.run.smithi012.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 11 MB/s | 713 kB 00:00 2023-07-08T12:35:26.625 INFO:teuthology.orchestra.run.smithi012.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 530 kB/s | 27 kB 00:00 2023-07-08T12:35:26.675 INFO:teuthology.orchestra.run.smithi012.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-07-08T12:35:26.742 INFO:teuthology.orchestra.run.smithi012.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 24 MB/s | 4.8 MB 00:00 2023-07-08T12:35:26.759 INFO:teuthology.orchestra.run.smithi012.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 526 kB/s | 44 kB 00:00 2023-07-08T12:35:26.809 INFO:teuthology.orchestra.run.smithi012.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 8.9 MB/s | 457 kB 00:00 2023-07-08T12:35:27.068 INFO:teuthology.orchestra.run.smithi012.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 11 MB/s | 3.7 MB 00:00 2023-07-08T12:35:27.085 INFO:teuthology.orchestra.run.smithi012.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 817 kB/s | 225 kB 00:00 2023-07-08T12:35:27.111 INFO:teuthology.orchestra.run.smithi012.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-07-08T12:35:27.136 INFO:teuthology.orchestra.run.smithi012.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-07-08T12:35:27.161 INFO:teuthology.orchestra.run.smithi012.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-07-08T12:35:27.187 INFO:teuthology.orchestra.run.smithi012.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-08T12:35:27.237 INFO:teuthology.orchestra.run.smithi012.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-07-08T12:35:27.287 INFO:teuthology.orchestra.run.smithi012.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-07-08T12:35:27.396 INFO:teuthology.orchestra.run.smithi012.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 23 MB/s | 4.9 MB 00:00 2023-07-08T12:35:27.438 INFO:teuthology.orchestra.run.smithi012.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-08T12:35:27.514 INFO:teuthology.orchestra.run.smithi012.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-07-08T12:35:27.538 INFO:teuthology.orchestra.run.smithi012.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.4 MB/s | 354 kB 00:00 2023-07-08T12:35:27.588 INFO:teuthology.orchestra.run.smithi012.stdout:(125/131): postgresql-contrib-10.21-2.module+el 16 MB/s | 808 kB 00:00 2023-07-08T12:35:27.656 INFO:teuthology.orchestra.run.smithi012.stdout:(126/131): postgresql-10.21-2.module+el8.6.0+15 23 MB/s | 1.5 MB 00:00 2023-07-08T12:35:27.756 INFO:teuthology.orchestra.run.smithi012.stdout:(127/131): postgresql-server-10.21-2.module+el8 21 MB/s | 5.1 MB 00:00 2023-07-08T12:35:28.298 INFO:teuthology.orchestra.run.smithi012.stdout:(128/131): ceph-test-18.0.0-4738.gf3a06ccd.el8. 4.9 MB/s | 48 MB 00:09 2023-07-08T12:35:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:(129/131): librados2-18.0.0-4738.gf3a06ccd.el8. 5.0 MB/s | 3.7 MB 00:00 2023-07-08T12:35:28.448 INFO:teuthology.orchestra.run.smithi012.stdout:(130/131): librbd1-18.0.0-4738.gf3a06ccd.el8.x8 5.1 MB/s | 3.6 MB 00:00 2023-07-08T12:35:28.490 INFO:teuthology.orchestra.run.smithi012.stdout:(131/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.1 MB/s | 798 kB 00:00 2023-07-08T12:35:28.494 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2023-07-08T12:35:28.495 INFO:teuthology.orchestra.run.smithi012.stdout:Total 16 MB/s | 205 MB 00:13 2023-07-08T12:35:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2023-07-08T12:35:29.332 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2023-07-08T12:35:29.363 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-08T12:35:29.464 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2023-07-08T12:35:29.465 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2023-07-08T12:35:29.543 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-08T12:35:29.780 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-08T12:35:29.948 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-08T12:35:30.076 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-08T12:35:30.257 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-08T12:35:30.645 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2023-07-08T12:35:30.650 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2023-07-08T12:35:30.661 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-08T12:35:30.865 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librados2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 6/134 2023-07-08T12:35:31.257 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 6/134 2023-07-08T12:35:31.457 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librbd1-2:18.0.0-4738.gf3a06ccd.el8.x86_64 7/134 2023-07-08T12:35:31.565 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:18.0.0-4738.gf3a06ccd.el8.x86_64 7/134 2023-07-08T12:35:31.740 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 8/134 2023-07-08T12:35:32.130 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephfs2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 8/134 2023-07-08T12:35:32.340 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rados-2:18.0.0-4738.gf3a06ccd.el8.x86_64 9/134 2023-07-08T12:35:32.566 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-08T12:35:32.785 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-08T12:35:33.166 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-08T12:35:33.393 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-08T12:35:33.598 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephsqlite-2:18.0.0-4738.gf3a06ccd.el8.x86_64 14/134 2023-07-08T12:35:33.692 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4738.gf3a06ccd.el8.x86_64 14/134 2023-07-08T12:35:33.864 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libradosstriper1-2:18.0.0-4738.gf3a06ccd.el8.x86 15/134 2023-07-08T12:35:33.924 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4738.gf3a06ccd.el8.x86 15/134 2023-07-08T12:35:34.048 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-08T12:35:34.191 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-08T12:35:34.364 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-08T12:35:34.984 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-08T12:35:35.164 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-08T12:35:35.767 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-08T12:35:35.989 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-08T12:35:35.990 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2023-07-08T12:35:36.193 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-08T12:35:36.386 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-08T12:35:36.580 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-08T12:35:36.690 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-08T12:35:36.707 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-08T12:35:36.932 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-08T12:35:36.973 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-08T12:35:37.216 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-08T12:35:37.361 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-08T12:35:37.594 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librados2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 5/134 2023-07-08T12:35:37.695 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-08T12:35:37.889 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-08T12:35:37.996 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 5/134 2023-07-08T12:35:38.084 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-08T12:35:38.336 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librbd1-2:18.0.0-4738.gf3a06ccd.el8.x86_64 6/134 2023-07-08T12:35:38.343 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-08T12:35:38.454 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:18.0.0-4738.gf3a06ccd.el8.x86_64 6/134 2023-07-08T12:35:38.521 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-common-2:18.0.0-4738.gf3a06ccd.el8. 26/134 2023-07-08T12:35:38.643 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-argparse-2:18.0.0-4738.gf3a06ccd.el 27/134 2023-07-08T12:35:38.644 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 7/134 2023-07-08T12:35:38.732 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephfs2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 7/134 2023-07-08T12:35:38.933 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rados-2:18.0.0-4738.gf3a06ccd.el8.x86_64 8/134 2023-07-08T12:35:39.138 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-08T12:35:39.413 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-08T12:35:39.560 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cephfs-2:18.0.0-4738.gf3a06ccd.el8.x86_6 28/134 2023-07-08T12:35:39.665 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-08T12:35:39.911 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-08T12:35:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-08T12:35:40.167 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-08T12:35:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-08T12:35:40.436 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-08T12:35:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephsqlite-2:18.0.0-4738.gf3a06ccd.el8.x86_64 14/134 2023-07-08T12:35:40.622 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-08T12:35:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4738.gf3a06ccd.el8.x86_64 14/134 2023-07-08T12:35:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libradosstriper1-2:18.0.0-4738.gf3a06ccd.el8.x86 15/134 2023-07-08T12:35:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4738.gf3a06ccd.el8.x86 15/134 2023-07-08T12:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-08T12:35:41.367 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-08T12:35:41.569 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-08T12:35:41.685 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-08T12:35:41.834 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-08T12:35:41.993 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-08T12:35:42.388 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-08T12:35:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-08T12:35:42.690 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-08T12:35:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-08T12:35:42.933 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-08T12:35:43.423 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-08T12:35:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-08T12:35:43.619 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rbd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 36/134 2023-07-08T12:35:43.684 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-08T12:35:43.776 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librados-devel-2:18.0.0-4738.gf3a06ccd.el8.x86_6 37/134 2023-07-08T12:35:43.939 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-08T12:35:44.124 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-08T12:35:44.340 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-08T12:35:44.428 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-08T12:35:44.475 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-08T12:35:44.662 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-08T12:35:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-08T12:35:44.869 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-08T12:35:44.895 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-08T12:35:45.148 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-08T12:35:45.246 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-08T12:35:45.369 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-common-2:18.0.0-4738.gf3a06ccd.el8. 26/134 2023-07-08T12:35:45.524 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-08T12:35:45.531 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-argparse-2:18.0.0-4738.gf3a06ccd.el 27/134 2023-07-08T12:35:45.739 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-08T12:35:45.816 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cephfs-2:18.0.0-4738.gf3a06ccd.el8.x86_6 28/134 2023-07-08T12:35:45.968 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-08T12:35:46.034 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-08T12:35:46.204 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-08T12:35:46.521 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-08T12:35:46.787 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-08T12:35:46.964 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-08T12:35:47.161 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-08T12:35:47.360 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-08T12:35:47.551 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-08T12:35:47.868 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-08T12:35:47.929 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-08T12:35:48.112 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-08T12:35:48.369 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-08T12:35:48.594 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-08T12:35:48.745 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-08T12:35:48.915 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-08T12:35:49.060 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-08T12:35:49.087 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-08T12:35:49.162 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-08T12:35:49.399 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-08T12:35:49.414 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-08T12:35:49.616 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-08T12:35:49.697 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-08T12:35:49.804 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-08T12:35:49.866 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-08T12:35:49.920 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-08T12:35:50.078 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-08T12:35:50.156 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-08T12:35:50.256 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-08T12:35:50.354 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-08T12:35:50.471 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-08T12:35:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rbd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 36/134 2023-07-08T12:35:50.651 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-08T12:35:50.690 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librados-devel-2:18.0.0-4738.gf3a06ccd.el8.x86_6 37/134 2023-07-08T12:35:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-08T12:35:50.929 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-08T12:35:51.027 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-08T12:35:51.127 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-08T12:35:51.224 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-08T12:35:51.298 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-08T12:35:51.308 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-08T12:35:51.465 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-08T12:35:51.526 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-08T12:35:51.673 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-08T12:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-08T12:35:51.884 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-08T12:35:51.924 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-08T12:35:51.993 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-08T12:35:52.099 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-08T12:35:52.184 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-08T12:35:52.257 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-08T12:35:52.379 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-08T12:35:52.418 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-08T12:35:52.555 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-08T12:35:52.618 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-08T12:35:52.769 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-08T12:35:52.841 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-08T12:35:52.935 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-08T12:35:53.026 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-08T12:35:53.105 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-08T12:35:53.266 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-08T12:35:53.301 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-08T12:35:53.486 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-08T12:35:53.522 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-08T12:35:53.669 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-08T12:35:53.887 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-08T12:35:53.911 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-08T12:35:54.481 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-08T12:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-08T12:35:54.990 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-08T12:35:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-08T12:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-08T12:35:55.266 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-08T12:35:55.502 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-08T12:35:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-08T12:35:55.786 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-08T12:35:55.801 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-08T12:35:55.984 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-08T12:35:56.054 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-08T12:35:56.208 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-08T12:35:56.256 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-08T12:35:56.378 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-08T12:35:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-08T12:35:56.914 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-08T12:35:57.047 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-08T12:35:57.181 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-08T12:35:57.228 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librgw2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 92/134 2023-07-08T12:35:57.300 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librgw2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 92/134 2023-07-08T12:35:57.458 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-08T12:35:57.496 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 93/134 2023-07-08T12:35:57.725 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-08T12:35:57.890 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-08T12:35:58.119 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-08T12:35:58.403 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-08T12:35:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-08T12:35:58.811 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-08T12:35:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-08T12:35:59.211 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-08T12:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-08T12:35:59.617 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-08T12:35:59.842 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-08T12:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-08T12:36:00.232 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-08T12:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-08T12:36:00.637 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-08T12:36:00.893 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-08T12:36:01.136 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:18.0.0-4738.gf3a06ccd.el8.x86_64 94/134 2023-07-08T12:36:01.139 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-08T12:36:01.340 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-08T12:36:01.369 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-common-2:18.0.0-4738.gf3a06ccd.el8.x86_64 94/134 2023-07-08T12:36:01.544 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-08T12:36:01.952 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:18.0.0-4738.gf3a06ccd.el8.x86_64 94/134 2023-07-08T12:36:02.254 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-base-2:18.0.0-4738.gf3a06ccd.el8.x86_64 95/134 2023-07-08T12:36:02.325 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-base-2:18.0.0-4738.gf3a06ccd.el8.x86_64 95/134 2023-07-08T12:36:02.460 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-selinux-2:18.0.0-4738.gf3a06ccd.el8.x86_64 96/134 2023-07-08T12:36:02.718 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-08T12:36:03.003 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-08T12:36:03.275 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-08T12:36:03.572 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-08T12:36:03.849 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-08T12:36:04.068 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-08T12:36:04.956 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-08T12:36:05.146 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librgw2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 92/134 2023-07-08T12:36:05.210 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librgw2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 92/134 2023-07-08T12:36:05.372 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 93/134 2023-07-08T12:36:09.352 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:18.0.0-4738.gf3a06ccd.el8.x86_64 94/134 2023-07-08T12:36:09.571 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-common-2:18.0.0-4738.gf3a06ccd.el8.x86_64 94/134 2023-07-08T12:36:10.202 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:18.0.0-4738.gf3a06ccd.el8.x86_64 94/134 2023-07-08T12:36:10.712 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-base-2:18.0.0-4738.gf3a06ccd.el8.x86_64 95/134 2023-07-08T12:36:10.797 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-base-2:18.0.0-4738.gf3a06ccd.el8.x86_64 95/134 2023-07-08T12:36:10.988 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-selinux-2:18.0.0-4738.gf3a06ccd.el8.x86_64 96/134 2023-07-08T12:36:21.085 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4738.gf3a06ccd.el8.x86_64 96/134 2023-07-08T12:36:21.086 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /sys 2023-07-08T12:36:21.086 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /proc 2023-07-08T12:36:21.086 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /dev 2023-07-08T12:36:21.086 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /run 2023-07-08T12:36:21.086 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /mnt 2023-07-08T12:36:21.086 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /var/tmp 2023-07-08T12:36:21.087 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /home 2023-07-08T12:36:21.087 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /tmp 2023-07-08T12:36:21.087 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /dev 2023-07-08T12:36:21.087 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:36:21.433 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-volume-2:18.0.0-4738.gf3a06ccd.el8.noarch 97/134 2023-07-08T12:36:23.007 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-volume-2:18.0.0-4738.gf3a06ccd.el8.noarch 97/134 2023-07-08T12:36:23.167 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-osd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 98/134 2023-07-08T12:36:23.386 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-osd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 98/134 2023-07-08T12:36:23.541 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mds-2:18.0.0-4738.gf3a06ccd.el8.x86_64 99/134 2023-07-08T12:36:23.996 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mds-2:18.0.0-4738.gf3a06ccd.el8.x86_64 99/134 2023-07-08T12:36:24.190 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mon-2:18.0.0-4738.gf3a06ccd.el8.x86_64 100/134 2023-07-08T12:36:24.207 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mon-2:18.0.0-4738.gf3a06ccd.el8.x86_64 100/134 2023-07-08T12:36:25.824 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephadm-2:18.0.0-4738.gf3a06ccd.el8.noarch 101/134 2023-07-08T12:36:26.044 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephadm-2:18.0.0-4738.gf3a06ccd.el8.noarch 101/134 2023-07-08T12:36:26.171 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4738.gf3a06ccd.e 102/134 2023-07-08T12:36:26.362 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4738.gf3a06ccd. 103/134 2023-07-08T12:36:26.542 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4738.gf3a06ccd.el8.noa 104/134 2023-07-08T12:36:26.755 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4738.gf3a06ccd.el8.noa 104/134 2023-07-08T12:36:27.083 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4738.gf3a06ccd.el8.n 105/134 2023-07-08T12:36:28.044 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4738.gf3a06ccd.el8.n 105/134 2023-07-08T12:36:28.189 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4738.gf3a 106/134 2023-07-08T12:36:28.280 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4738.gf3a 106/134 2023-07-08T12:36:28.653 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4738.gf3a06ccd.el 107/134 2023-07-08T12:36:28.836 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-2:18.0.0-4738.gf3a06ccd.el8.x86_64 108/134 2023-07-08T12:36:28.869 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4738.gf3a06ccd.el8.x86_64 108/134 2023-07-08T12:36:29.031 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-rook-2:18.0.0-4738.gf3a06ccd.el8.noarch 109/134 2023-07-08T12:36:29.042 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4738.gf3a06ccd.el8.noarch 109/134 2023-07-08T12:36:30.352 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-2:18.0.0-4738.gf3a06ccd.el8.x86_64 110/134 2023-07-08T12:36:30.562 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4738.gf3a06ccd.el8.x86_64 96/134 2023-07-08T12:36:30.563 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /sys 2023-07-08T12:36:30.563 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /proc 2023-07-08T12:36:30.563 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /dev 2023-07-08T12:36:30.563 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /run 2023-07-08T12:36:30.563 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /mnt 2023-07-08T12:36:30.563 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /var/tmp 2023-07-08T12:36:30.563 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /home 2023-07-08T12:36:30.564 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /tmp 2023-07-08T12:36:30.564 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /dev 2023-07-08T12:36:30.564 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:36:30.595 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-radosgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 111/134 2023-07-08T12:36:30.629 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 111/134 2023-07-08T12:36:30.781 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4738.gf3a06 112/134 2023-07-08T12:36:30.824 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4738.gf3a06 112/134 2023-07-08T12:36:30.899 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-volume-2:18.0.0-4738.gf3a06ccd.el8.noarch 97/134 2023-07-08T12:36:30.998 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 113/134 2023-07-08T12:36:31.344 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 113/134 2023-07-08T12:36:31.514 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 114/134 2023-07-08T12:36:32.582 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-volume-2:18.0.0-4738.gf3a06ccd.el8.noarch 97/134 2023-07-08T12:36:32.741 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-osd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 98/134 2023-07-08T12:36:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-osd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 98/134 2023-07-08T12:36:33.107 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mds-2:18.0.0-4738.gf3a06ccd.el8.x86_64 99/134 2023-07-08T12:36:33.589 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mds-2:18.0.0-4738.gf3a06ccd.el8.x86_64 99/134 2023-07-08T12:36:33.790 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mon-2:18.0.0-4738.gf3a06ccd.el8.x86_64 100/134 2023-07-08T12:36:33.807 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mon-2:18.0.0-4738.gf3a06ccd.el8.x86_64 100/134 2023-07-08T12:36:35.687 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephadm-2:18.0.0-4738.gf3a06ccd.el8.noarch 101/134 2023-07-08T12:36:35.868 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephadm-2:18.0.0-4738.gf3a06ccd.el8.noarch 101/134 2023-07-08T12:36:36.012 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4738.gf3a06ccd.e 102/134 2023-07-08T12:36:36.035 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 114/134 2023-07-08T12:36:36.173 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4738.gf3a06ccd. 103/134 2023-07-08T12:36:36.313 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-test-2:18.0.0-4738.gf3a06ccd.el8.x86_64 115/134 2023-07-08T12:36:36.364 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4738.gf3a06ccd.el8.noa 104/134 2023-07-08T12:36:36.471 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-08T12:36:36.587 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4738.gf3a06ccd.el8.noa 104/134 2023-07-08T12:36:36.671 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-08T12:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4738.gf3a06ccd.el8.n 105/134 2023-07-08T12:36:37.597 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-08T12:36:37.921 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4738.gf3a06ccd.el8.n 105/134 2023-07-08T12:36:37.927 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-08T12:36:38.072 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4738.gf3a 106/134 2023-07-08T12:36:38.108 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-fuse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 119/134 2023-07-08T12:36:38.167 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4738.gf3a 106/134 2023-07-08T12:36:38.325 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-08T12:36:38.508 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs-devel-2:18.0.0-4738.gf3a06ccd.el8.x86_ 121/134 2023-07-08T12:36:38.568 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4738.gf3a06ccd.el 107/134 2023-07-08T12:36:38.656 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-08T12:36:38.789 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-2:18.0.0-4738.gf3a06ccd.el8.x86_64 108/134 2023-07-08T12:36:38.826 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4738.gf3a06ccd.el8.x86_64 108/134 2023-07-08T12:36:38.980 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-rook-2:18.0.0-4738.gf3a06ccd.el8.noarch 109/134 2023-07-08T12:36:38.991 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4738.gf3a06ccd.el8.noarch 109/134 2023-07-08T12:36:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-2:18.0.0-4738.gf3a06ccd.el8.x86_64 110/134 2023-07-08T12:36:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-radosgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 111/134 2023-07-08T12:36:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 111/134 2023-07-08T12:36:40.787 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-08T12:36:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4738.gf3a06 112/134 2023-07-08T12:36:41.004 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-08T12:36:41.022 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-08T12:36:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4738.gf3a06 112/134 2023-07-08T12:36:41.179 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-top-2:18.0.0-4738.gf3a06ccd.el8.noarch 124/134 2023-07-08T12:36:41.180 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 113/134 2023-07-08T12:36:41.470 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-fuse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 125/134 2023-07-08T12:36:41.550 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 113/134 2023-07-08T12:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 114/134 2023-07-08T12:36:41.739 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-nbd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 126/134 2023-07-08T12:36:42.031 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-08T12:36:42.196 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-08T12:36:42.204 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-08T12:36:42.384 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-08T12:36:42.562 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-08T12:36:42.645 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-08T12:36:42.837 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-08T12:36:42.850 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-08T12:36:42.850 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-08T12:36:43.073 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-08T12:36:43.073 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-08T12:36:43.267 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-08T12:36:43.436 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-08T12:36:43.954 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-08T12:36:43.955 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-2:18.0.0-4738.gf3a06ccd.el8.x86_64 1/134 2023-07-08T12:36:43.955 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-base-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2/134 2023-07-08T12:36:43.955 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-common-2:18.0.0-4738.gf3a06ccd.el8.x86_64 3/134 2023-07-08T12:36:43.956 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-fuse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 4/134 2023-07-08T12:36:43.956 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4738.gf3a06 5/134 2023-07-08T12:36:43.956 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mds-2:18.0.0-4738.gf3a06ccd.el8.x86_64 6/134 2023-07-08T12:36:43.956 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-2:18.0.0-4738.gf3a06ccd.el8.x86_64 7/134 2023-07-08T12:36:43.957 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mon-2:18.0.0-4738.gf3a06ccd.el8.x86_64 8/134 2023-07-08T12:36:43.957 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-osd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 9/134 2023-07-08T12:36:43.957 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-radosgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 10/134 2023-07-08T12:36:43.958 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-selinux-2:18.0.0-4738.gf3a06ccd.el8.x86_64 11/134 2023-07-08T12:36:43.958 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-test-2:18.0.0-4738.gf3a06ccd.el8.x86_64 12/134 2023-07-08T12:36:43.958 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 13/134 2023-07-08T12:36:43.959 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs-devel-2:18.0.0-4738.gf3a06ccd.el8.x86_ 14/134 2023-07-08T12:36:43.959 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 15/134 2023-07-08T12:36:43.959 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephsqlite-2:18.0.0-4738.gf3a06ccd.el8.x86_64 16/134 2023-07-08T12:36:43.959 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados-devel-2:18.0.0-4738.gf3a06ccd.el8.x86_6 17/134 2023-07-08T12:36:43.959 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libradosstriper1-2:18.0.0-4738.gf3a06ccd.el8.x86 18/134 2023-07-08T12:36:43.960 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librgw2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 19/134 2023-07-08T12:36:43.960 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4738.gf3a06ccd.el 20/134 2023-07-08T12:36:43.960 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-common-2:18.0.0-4738.gf3a06ccd.el8. 21/134 2023-07-08T12:36:43.960 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cephfs-2:18.0.0-4738.gf3a06ccd.el8.x86_6 22/134 2023-07-08T12:36:43.961 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rados-2:18.0.0-4738.gf3a06ccd.el8.x86_64 23/134 2023-07-08T12:36:43.961 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rbd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 24/134 2023-07-08T12:36:43.961 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 25/134 2023-07-08T12:36:43.961 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-fuse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 26/134 2023-07-08T12:36:43.962 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 27/134 2023-07-08T12:36:43.962 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-nbd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 28/134 2023-07-08T12:36:43.963 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4738.gf3a06ccd. 29/134 2023-07-08T12:36:43.963 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4738.gf3a06ccd.el8.noa 30/134 2023-07-08T12:36:43.963 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4738.gf3a06ccd.el8.n 31/134 2023-07-08T12:36:43.963 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4738.gf3a 32/134 2023-07-08T12:36:43.963 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4738.gf3a06ccd.el 33/134 2023-07-08T12:36:43.963 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4738.gf3a06ccd.el8.noarch 34/134 2023-07-08T12:36:43.963 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4738.gf3a06ccd.e 35/134 2023-07-08T12:36:43.964 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-volume-2:18.0.0-4738.gf3a06ccd.el8.noarch 36/134 2023-07-08T12:36:43.964 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephadm-2:18.0.0-4738.gf3a06ccd.el8.noarch 37/134 2023-07-08T12:36:43.964 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-top-2:18.0.0-4738.gf3a06ccd.el8.noarch 38/134 2023-07-08T12:36:43.964 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-08T12:36:43.964 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-08T12:36:43.964 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-08T12:36:43.964 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-08T12:36:43.965 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-08T12:36:43.965 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-08T12:36:43.965 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-08T12:36:43.965 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-08T12:36:43.965 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-08T12:36:43.965 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-08T12:36:43.965 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-08T12:36:43.966 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-08T12:36:43.966 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-08T12:36:43.966 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-08T12:36:43.966 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-08T12:36:43.966 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-08T12:36:43.966 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-08T12:36:43.966 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-08T12:36:43.967 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-08T12:36:43.967 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-08T12:36:43.967 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-08T12:36:43.967 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-08T12:36:43.967 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-08T12:36:43.967 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-08T12:36:43.967 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-08T12:36:43.967 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-08T12:36:43.968 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-08T12:36:43.968 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-08T12:36:43.968 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-08T12:36:43.968 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-08T12:36:43.968 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-08T12:36:43.968 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-08T12:36:43.968 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-08T12:36:43.968 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-08T12:36:43.969 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-08T12:36:43.969 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-08T12:36:43.969 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-08T12:36:43.969 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-08T12:36:43.969 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-08T12:36:43.969 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-08T12:36:43.969 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-08T12:36:43.970 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-08T12:36:43.970 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-08T12:36:43.970 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-08T12:36:43.970 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-08T12:36:43.970 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-08T12:36:43.970 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-08T12:36:43.970 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-08T12:36:43.970 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-08T12:36:43.971 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-08T12:36:43.971 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-08T12:36:43.971 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-08T12:36:43.971 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-08T12:36:43.971 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-08T12:36:43.971 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-08T12:36:43.971 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-08T12:36:43.971 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-08T12:36:43.972 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-08T12:36:43.972 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-08T12:36:43.972 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-08T12:36:43.972 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-08T12:36:43.972 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-08T12:36:43.972 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-08T12:36:43.972 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-08T12:36:43.973 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-08T12:36:43.973 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-08T12:36:43.973 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-08T12:36:43.973 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-08T12:36:43.973 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-08T12:36:43.973 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-08T12:36:43.973 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-08T12:36:43.974 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-08T12:36:43.974 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-08T12:36:43.974 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-08T12:36:43.974 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-08T12:36:43.974 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-08T12:36:43.974 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-08T12:36:43.974 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-08T12:36:43.974 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-08T12:36:43.975 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-08T12:36:43.975 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-08T12:36:43.975 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-08T12:36:43.975 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-08T12:36:43.975 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-08T12:36:43.975 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-08T12:36:43.975 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-08T12:36:43.975 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-08T12:36:43.976 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-08T12:36:43.976 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-08T12:36:43.976 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-08T12:36:43.976 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 129/134 2023-07-08T12:36:43.976 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-08T12:36:43.976 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:18.0.0-4738.gf3a06ccd.el8.x86_64 131/134 2023-07-08T12:36:43.976 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-08T12:36:43.976 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-08T12:36:46.586 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 114/134 2023-07-08T12:36:46.690 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-08T12:36:46.691 INFO:teuthology.orchestra.run.smithi160.stdout:Last metadata expiration check: 0:01:47 ago on Sat 08 Jul 2023 12:34:59 PM UTC. 2023-07-08T12:36:47.566 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-test-2:18.0.0-4738.gf3a06ccd.el8.x86_64 115/134 2023-07-08T12:36:47.915 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-08T12:36:48.097 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-fuse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 117/134 2023-07-08T12:36:48.346 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-08T12:36:48.520 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-08T12:36:48.724 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-08T12:36:48.991 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-08T12:36:49.141 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs-devel-2:18.0.0-4738.gf3a06ccd.el8.x86_ 121/134 2023-07-08T12:36:49.313 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-08T12:36:50.097 INFO:teuthology.orchestra.run.smithi160.stdout:Installed products updated. 2023-07-08T12:36:50.321 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:36:50.321 INFO:teuthology.orchestra.run.smithi160.stdout:Upgraded: 2023-07-08T12:36:50.321 INFO:teuthology.orchestra.run.smithi160.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-08T12:36:50.321 INFO:teuthology.orchestra.run.smithi160.stdout: librados2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.322 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.322 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2023-07-08T12:36:50.322 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-08T12:36:50.322 INFO:teuthology.orchestra.run.smithi160.stdout: automake-1.16.1-7.el8.noarch 2023-07-08T12:36:50.322 INFO:teuthology.orchestra.run.smithi160.stdout: bison-3.0.4-10.el8.x86_64 2023-07-08T12:36:50.322 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.322 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.323 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.323 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.323 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:36:50.323 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.323 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.323 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.323 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:36:50.323 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:36:50.324 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:36:50.324 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:36:50.324 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:36:50.324 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.324 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.324 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:36:50.324 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.325 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.325 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.325 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:36:50.325 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:36:50.325 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.325 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:36:50.325 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-08T12:36:50.325 INFO:teuthology.orchestra.run.smithi160.stdout: flex-2.6.1-9.el8.x86_64 2023-07-08T12:36:50.326 INFO:teuthology.orchestra.run.smithi160.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-08T12:36:50.326 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-08T12:36:50.326 INFO:teuthology.orchestra.run.smithi160.stdout: jq-1.6-3.el8.x86_64 2023-07-08T12:36:50.326 INFO:teuthology.orchestra.run.smithi160.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-08T12:36:50.326 INFO:teuthology.orchestra.run.smithi160.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-08T12:36:50.326 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-08T12:36:50.326 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.327 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.327 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.327 INFO:teuthology.orchestra.run.smithi160.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-08T12:36:50.327 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-08T12:36:50.327 INFO:teuthology.orchestra.run.smithi160.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-08T12:36:50.327 INFO:teuthology.orchestra.run.smithi160.stdout: libpq-13.5-1.el8.x86_64 2023-07-08T12:36:50.327 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-08T12:36:50.327 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-08T12:36:50.328 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.328 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.328 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-08T12:36:50.328 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.328 INFO:teuthology.orchestra.run.smithi160.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-08T12:36:50.328 INFO:teuthology.orchestra.run.smithi160.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-08T12:36:50.328 INFO:teuthology.orchestra.run.smithi160.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-08T12:36:50.329 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-08T12:36:50.329 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-08T12:36:50.329 INFO:teuthology.orchestra.run.smithi160.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-08T12:36:50.329 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-08T12:36:50.329 INFO:teuthology.orchestra.run.smithi160.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-08T12:36:50.329 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-08T12:36:50.329 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-08T12:36:50.330 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-08T12:36:50.331 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-08T12:36:50.331 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-08T12:36:50.331 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-08T12:36:50.331 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-08T12:36:50.331 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-08T12:36:50.331 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-08T12:36:50.332 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-08T12:36:50.332 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-08T12:36:50.332 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-08T12:36:50.332 INFO:teuthology.orchestra.run.smithi160.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-08T12:36:50.332 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-08T12:36:50.332 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.332 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.332 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.333 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-08T12:36:50.333 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-08T12:36:50.333 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-08T12:36:50.333 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-08T12:36:50.333 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-08T12:36:50.333 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-08T12:36:50.333 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-08T12:36:50.334 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-08T12:36:50.334 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-08T12:36:50.334 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-08T12:36:50.334 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-08T12:36:50.334 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-08T12:36:50.334 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-08T12:36:50.334 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-08T12:36:50.334 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-08T12:36:50.335 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-08T12:36:50.335 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-08T12:36:50.335 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-08T12:36:50.335 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-08T12:36:50.335 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-08T12:36:50.335 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-08T12:36:50.335 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-08T12:36:50.335 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend-2.6-1.el8.noarch 2023-07-08T12:36:50.336 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-08T12:36:50.336 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-08T12:36:50.336 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-08T12:36:50.336 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-08T12:36:50.336 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-08T12:36:50.336 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-08T12:36:50.336 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.336 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.337 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-08T12:36:50.337 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-08T12:36:50.337 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.337 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-08T12:36:50.337 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-08T12:36:50.337 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-08T12:36:50.337 INFO:teuthology.orchestra.run.smithi160.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-08T12:36:50.338 INFO:teuthology.orchestra.run.smithi160.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-08T12:36:50.338 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-08T12:36:50.338 INFO:teuthology.orchestra.run.smithi160.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-08T12:36:50.338 INFO:teuthology.orchestra.run.smithi160.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-08T12:36:50.338 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-08T12:36:50.338 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-08T12:36:50.338 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-08T12:36:50.338 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-08T12:36:50.339 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-08T12:36:50.339 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.339 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.339 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:36:50.339 INFO:teuthology.orchestra.run.smithi160.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-08T12:36:50.339 INFO:teuthology.orchestra.run.smithi160.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-08T12:36:50.339 INFO:teuthology.orchestra.run.smithi160.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-08T12:36:50.339 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-08T12:36:50.340 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-08T12:36:50.340 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-08T12:36:50.340 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:36:50.341 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2023-07-08T12:36:50.534 DEBUG:teuthology.parallel:result is None 2023-07-08T12:36:51.682 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-08T12:36:51.936 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-08T12:36:51.954 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-08T12:36:52.103 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-top-2:18.0.0-4738.gf3a06ccd.el8.noarch 124/134 2023-07-08T12:36:52.437 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-fuse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 125/134 2023-07-08T12:36:52.721 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-nbd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 126/134 2023-07-08T12:36:53.171 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-08T12:36:53.352 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-08T12:36:53.473 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-08T12:36:53.694 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-08T12:36:53.708 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-08T12:36:53.902 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-08T12:36:53.912 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-08T12:36:54.046 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-08T12:36:54.047 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-08T12:36:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-08T12:36:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-08T12:36:54.659 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-08T12:36:54.826 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-08T12:36:55.375 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-08T12:36:55.375 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-2:18.0.0-4738.gf3a06ccd.el8.x86_64 1/134 2023-07-08T12:36:55.376 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-base-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2/134 2023-07-08T12:36:55.376 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-common-2:18.0.0-4738.gf3a06ccd.el8.x86_64 3/134 2023-07-08T12:36:55.376 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-fuse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 4/134 2023-07-08T12:36:55.376 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4738.gf3a06 5/134 2023-07-08T12:36:55.377 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mds-2:18.0.0-4738.gf3a06ccd.el8.x86_64 6/134 2023-07-08T12:36:55.377 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-2:18.0.0-4738.gf3a06ccd.el8.x86_64 7/134 2023-07-08T12:36:55.377 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mon-2:18.0.0-4738.gf3a06ccd.el8.x86_64 8/134 2023-07-08T12:36:55.377 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-osd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 9/134 2023-07-08T12:36:55.377 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-radosgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 10/134 2023-07-08T12:36:55.378 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-selinux-2:18.0.0-4738.gf3a06ccd.el8.x86_64 11/134 2023-07-08T12:36:55.378 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-test-2:18.0.0-4738.gf3a06ccd.el8.x86_64 12/134 2023-07-08T12:36:55.378 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 13/134 2023-07-08T12:36:55.378 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs-devel-2:18.0.0-4738.gf3a06ccd.el8.x86_ 14/134 2023-07-08T12:36:55.378 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 15/134 2023-07-08T12:36:55.379 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephsqlite-2:18.0.0-4738.gf3a06ccd.el8.x86_64 16/134 2023-07-08T12:36:55.379 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados-devel-2:18.0.0-4738.gf3a06ccd.el8.x86_6 17/134 2023-07-08T12:36:55.379 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libradosstriper1-2:18.0.0-4738.gf3a06ccd.el8.x86 18/134 2023-07-08T12:36:55.379 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librgw2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 19/134 2023-07-08T12:36:55.379 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4738.gf3a06ccd.el 20/134 2023-07-08T12:36:55.380 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-common-2:18.0.0-4738.gf3a06ccd.el8. 21/134 2023-07-08T12:36:55.380 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cephfs-2:18.0.0-4738.gf3a06ccd.el8.x86_6 22/134 2023-07-08T12:36:55.380 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rados-2:18.0.0-4738.gf3a06ccd.el8.x86_64 23/134 2023-07-08T12:36:55.380 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rbd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 24/134 2023-07-08T12:36:55.381 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 25/134 2023-07-08T12:36:55.381 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-fuse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 26/134 2023-07-08T12:36:55.381 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 27/134 2023-07-08T12:36:55.381 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-nbd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 28/134 2023-07-08T12:36:55.382 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4738.gf3a06ccd. 29/134 2023-07-08T12:36:55.382 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4738.gf3a06ccd.el8.noa 30/134 2023-07-08T12:36:55.382 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4738.gf3a06ccd.el8.n 31/134 2023-07-08T12:36:55.382 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4738.gf3a 32/134 2023-07-08T12:36:55.382 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4738.gf3a06ccd.el 33/134 2023-07-08T12:36:55.382 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4738.gf3a06ccd.el8.noarch 34/134 2023-07-08T12:36:55.382 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4738.gf3a06ccd.e 35/134 2023-07-08T12:36:55.383 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-volume-2:18.0.0-4738.gf3a06ccd.el8.noarch 36/134 2023-07-08T12:36:55.383 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephadm-2:18.0.0-4738.gf3a06ccd.el8.noarch 37/134 2023-07-08T12:36:55.383 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-top-2:18.0.0-4738.gf3a06ccd.el8.noarch 38/134 2023-07-08T12:36:55.383 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-08T12:36:55.383 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-08T12:36:55.383 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-08T12:36:55.384 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-08T12:36:55.384 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-08T12:36:55.384 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-08T12:36:55.384 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-08T12:36:55.384 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-08T12:36:55.384 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-08T12:36:55.385 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-08T12:36:55.385 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-08T12:36:55.385 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-08T12:36:55.385 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-08T12:36:55.386 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-08T12:36:55.386 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-08T12:36:55.386 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-08T12:36:55.387 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-08T12:36:55.387 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-08T12:36:55.387 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-08T12:36:55.387 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-08T12:36:55.388 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-08T12:36:55.388 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-08T12:36:55.388 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-08T12:36:55.389 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-08T12:36:55.389 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-08T12:36:55.389 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-08T12:36:55.389 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-08T12:36:55.390 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-08T12:36:55.390 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-08T12:36:55.390 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-08T12:36:55.391 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-08T12:36:55.391 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-08T12:36:55.391 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-08T12:36:55.391 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-08T12:36:55.391 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-08T12:36:55.392 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-08T12:36:55.392 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-08T12:36:55.392 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-08T12:36:55.393 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-08T12:36:55.393 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-08T12:36:55.393 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-08T12:36:55.394 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-08T12:36:55.394 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-08T12:36:55.394 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-08T12:36:55.394 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-08T12:36:55.395 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-08T12:36:55.395 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-08T12:36:55.395 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-08T12:36:55.395 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-08T12:36:55.396 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-08T12:36:55.396 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-08T12:36:55.396 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-08T12:36:55.396 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-08T12:36:55.397 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-08T12:36:55.397 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-08T12:36:55.397 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-08T12:36:55.398 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-08T12:36:55.398 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-08T12:36:55.398 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-08T12:36:55.398 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-08T12:36:55.399 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-08T12:36:55.399 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-08T12:36:55.399 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-08T12:36:55.399 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-08T12:36:55.400 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-08T12:36:55.400 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-08T12:36:55.400 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-08T12:36:55.401 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-08T12:36:55.401 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-08T12:36:55.401 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-08T12:36:55.401 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-08T12:36:55.402 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-08T12:36:55.402 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-08T12:36:55.402 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-08T12:36:55.403 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-08T12:36:55.403 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-08T12:36:55.403 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-08T12:36:55.403 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-08T12:36:55.404 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-08T12:36:55.404 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-08T12:36:55.404 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-08T12:36:55.405 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-08T12:36:55.405 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-08T12:36:55.405 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-08T12:36:55.405 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-08T12:36:55.406 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-08T12:36:55.406 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-08T12:36:55.406 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-08T12:36:55.406 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-08T12:36:55.407 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-08T12:36:55.407 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 129/134 2023-07-08T12:36:55.407 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-08T12:36:55.407 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:18.0.0-4738.gf3a06ccd.el8.x86_64 131/134 2023-07-08T12:36:55.408 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-08T12:36:55.408 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-08T12:36:58.066 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-08T12:36:58.066 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:56 ago on Sat 08 Jul 2023 12:35:02 PM UTC. 2023-07-08T12:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout:Installed products updated. 2023-07-08T12:37:03.491 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:03.491 INFO:teuthology.orchestra.run.smithi012.stdout:Upgraded: 2023-07-08T12:37:03.491 INFO:teuthology.orchestra.run.smithi012.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-08T12:37:03.491 INFO:teuthology.orchestra.run.smithi012.stdout: librados2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.492 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.492 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2023-07-08T12:37:03.492 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-08T12:37:03.492 INFO:teuthology.orchestra.run.smithi012.stdout: automake-1.16.1-7.el8.noarch 2023-07-08T12:37:03.492 INFO:teuthology.orchestra.run.smithi012.stdout: bison-3.0.4-10.el8.x86_64 2023-07-08T12:37:03.492 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.492 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.493 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.493 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.493 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:37:03.493 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.493 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.493 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.493 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:37:03.493 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:37:03.494 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:37:03.494 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:37:03.494 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:37:03.494 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.494 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.494 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:37:03.494 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.494 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.495 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.495 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:37:03.495 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:37:03.495 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.495 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top-2:18.0.0-4738.gf3a06ccd.el8.noarch 2023-07-08T12:37:03.495 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-08T12:37:03.495 INFO:teuthology.orchestra.run.smithi012.stdout: flex-2.6.1-9.el8.x86_64 2023-07-08T12:37:03.495 INFO:teuthology.orchestra.run.smithi012.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-08T12:37:03.496 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-08T12:37:03.496 INFO:teuthology.orchestra.run.smithi012.stdout: jq-1.6-3.el8.x86_64 2023-07-08T12:37:03.496 INFO:teuthology.orchestra.run.smithi012.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-08T12:37:03.496 INFO:teuthology.orchestra.run.smithi012.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-08T12:37:03.496 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-08T12:37:03.496 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.496 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.497 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.497 INFO:teuthology.orchestra.run.smithi012.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-08T12:37:03.497 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-08T12:37:03.497 INFO:teuthology.orchestra.run.smithi012.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-08T12:37:03.497 INFO:teuthology.orchestra.run.smithi012.stdout: libpq-13.5-1.el8.x86_64 2023-07-08T12:37:03.497 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-08T12:37:03.497 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-08T12:37:03.498 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.498 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.498 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-08T12:37:03.498 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.498 INFO:teuthology.orchestra.run.smithi012.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-08T12:37:03.498 INFO:teuthology.orchestra.run.smithi012.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-08T12:37:03.498 INFO:teuthology.orchestra.run.smithi012.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-08T12:37:03.498 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-08T12:37:03.499 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-08T12:37:03.499 INFO:teuthology.orchestra.run.smithi012.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-08T12:37:03.499 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-08T12:37:03.499 INFO:teuthology.orchestra.run.smithi012.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-08T12:37:03.499 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-08T12:37:03.499 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-08T12:37:03.499 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-08T12:37:03.499 INFO:teuthology.orchestra.run.smithi012.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-08T12:37:03.500 INFO:teuthology.orchestra.run.smithi012.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-08T12:37:03.500 INFO:teuthology.orchestra.run.smithi012.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-08T12:37:03.500 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-08T12:37:03.500 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-08T12:37:03.500 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-08T12:37:03.500 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-08T12:37:03.500 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-08T12:37:03.501 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-08T12:37:03.501 INFO:teuthology.orchestra.run.smithi012.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-08T12:37:03.501 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-08T12:37:03.501 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.501 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.501 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.501 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-08T12:37:03.501 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-08T12:37:03.502 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-08T12:37:03.502 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-08T12:37:03.502 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-08T12:37:03.502 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-08T12:37:03.502 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-08T12:37:03.502 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-08T12:37:03.502 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-08T12:37:03.502 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-08T12:37:03.503 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-08T12:37:03.503 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-08T12:37:03.503 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-08T12:37:03.503 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-08T12:37:03.503 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-08T12:37:03.503 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-08T12:37:03.503 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-08T12:37:03.503 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-08T12:37:03.504 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-08T12:37:03.504 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-08T12:37:03.504 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-08T12:37:03.504 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-08T12:37:03.504 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend-2.6-1.el8.noarch 2023-07-08T12:37:03.504 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-08T12:37:03.504 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-08T12:37:03.505 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-08T12:37:03.505 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-08T12:37:03.505 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-08T12:37:03.505 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-08T12:37:03.505 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.505 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.505 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-08T12:37:03.505 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-08T12:37:03.506 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.506 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-08T12:37:03.506 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-08T12:37:03.506 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-08T12:37:03.506 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-08T12:37:03.506 INFO:teuthology.orchestra.run.smithi012.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-08T12:37:03.506 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-08T12:37:03.506 INFO:teuthology.orchestra.run.smithi012.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-08T12:37:03.507 INFO:teuthology.orchestra.run.smithi012.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-08T12:37:03.507 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-08T12:37:03.507 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-08T12:37:03.507 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-08T12:37:03.507 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-08T12:37:03.507 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-08T12:37:03.507 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.507 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.508 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd-2:18.0.0-4738.gf3a06ccd.el8.x86_64 2023-07-08T12:37:03.508 INFO:teuthology.orchestra.run.smithi012.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-08T12:37:03.508 INFO:teuthology.orchestra.run.smithi012.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-08T12:37:03.508 INFO:teuthology.orchestra.run.smithi012.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-08T12:37:03.508 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-08T12:37:03.508 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-08T12:37:03.509 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-08T12:37:03.509 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:03.509 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2023-07-08T12:37:03.769 DEBUG:teuthology.parallel:result is None 2023-07-08T12:37:03.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f3a06ccd0ae55e9794bc8ce562926656b2738063 2023-07-08T12:37:03.989 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-08T12:37:04.013 INFO:teuthology.orchestra.run.smithi012.stdout:18.0.0-4738.gf3a06ccd.el8 2023-07-08T12:37:04.013 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4738.gf3a06ccd.el8 2023-07-08T12:37:04.014 INFO:teuthology.task.install:The correct ceph version 18.0.0-4738.gf3a06ccd is installed. 2023-07-08T12:37:04.014 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f3a06ccd0ae55e9794bc8ce562926656b2738063 2023-07-08T12:37:04.229 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-08T12:37:04.259 INFO:teuthology.orchestra.run.smithi160.stdout:18.0.0-4738.gf3a06ccd.el8 2023-07-08T12:37:04.259 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4738.gf3a06ccd.el8 2023-07-08T12:37:04.260 INFO:teuthology.task.install:The correct ceph version 18.0.0-4738.gf3a06ccd is installed. 2023-07-08T12:37:04.260 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-08T12:37:04.261 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:04.262 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-08T12:37:04.305 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:04.306 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-08T12:37:04.353 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-08T12:37:04.353 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:04.354 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2023-07-08T12:37:04.407 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-08T12:37:04.476 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:04.476 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/daemon-helper 2023-07-08T12:37:04.507 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-08T12:37:04.577 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-08T12:37:04.577 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:04.578 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-08T12:37:04.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-08T12:37:04.713 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:04.714 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-08T12:37:04.742 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-08T12:37:04.811 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-08T12:37:04.812 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:04.812 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2023-07-08T12:37:04.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-08T12:37:04.909 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:04.910 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/stdin-killer 2023-07-08T12:37:04.937 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-08T12:37:05.005 INFO:teuthology.run_tasks:Running task ceph... 2023-07-08T12:37:05.144 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-07-08T12:37:05.144 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /var/log/ceph 2023-07-08T12:37:05.148 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 777 /var/log/ceph 2023-07-08T12:37:05.175 INFO:tasks.ceph:Disabling ceph logrotate... 2023-07-08T12:37:05.176 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-08T12:37:05.214 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-08T12:37:05.241 INFO:tasks.ceph:Creating extra log directories... 2023-07-08T12:37:05.241 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-08T12:37:05.286 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-08T12:37:05.318 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-07-08T12:37:05.319 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-07-08T12:37:05.319 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-07-08T12:37:05.319 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:05.320 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-08T12:37:05.369 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-08T12:37:05.505 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-08T12:37:05.533 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:05.534 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-08T12:37:05.551 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-08T12:37:05.685 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-08T12:37:05.717 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-07-08T12:37:05.718 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-07-08T12:37:05.718 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-07-08_12:15:41-fs-wip-rishabh-mdscaps-2-testing-default-smithi/7329068', 'branch': 'wip-rishabh-mdscaps-2', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev}', 'email': None, 'first_in_suite': False, 'job_id': '7329068', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-07-08_12:15:41-fs-wip-rishabh-mdscaps-2-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-mdscaps-2'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'f3a06ccd0ae55e9794bc8ce562926656b2738063'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f3a06ccd0ae55e9794bc8ce562926656b2738063'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-mdscaps-2', 'sha1': 'f3a06ccd0ae55e9794bc8ce562926656b2738063'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': 'f3a06ccd0ae55e9794bc8ce562926656b2738063', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-mdscaps-2', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'f3a06ccd0ae55e9794bc8ce562926656b2738063', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi012.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi160.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f3a06ccd0ae55e9794bc8ce562926656b2738063'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b53f236c1077f1d295baafb9e0ca9a4b447019d7', 'timestamp': '2023-07-08_12:15:41', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-07-08T12:37:05.719 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-08T12:37:05.721 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-08T12:37:05.769 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/run/ceph 2023-07-08T12:37:05.786 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m0777 -- /var/run/ceph 2023-07-08T12:37:05.836 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:05.836 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2023-07-08T12:37:05.868 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-08T12:37:05.868 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2023-07-08T12:37:05.926 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-08T12:37:05.926 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-08T12:37:05.926 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 380980 Links: 1 2023-07-08T12:37:05.927 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-08T12:37:05.927 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2023-07-08T12:37:05.927 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-07-08 12:36:55.053781146 +0000 2023-07-08T12:37:05.927 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-07-08 12:34:10.680337611 +0000 2023-07-08T12:37:05.927 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-07-08 12:34:10.680337611 +0000 2023-07-08T12:37:05.927 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2023-07-08T12:37:05.928 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-08T12:37:05.993 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-07-08T12:37:05.993 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-07-08T12:37:05.994 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000230885 s, 2.2 MB/s 2023-07-08T12:37:05.995 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-08T12:37:06.054 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2023-07-08T12:37:06.111 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-08T12:37:06.112 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-08T12:37:06.112 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 364422 Links: 1 2023-07-08T12:37:06.113 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-08T12:37:06.113 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2023-07-08T12:37:06.113 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-07-08 12:36:55.053781146 +0000 2023-07-08T12:37:06.113 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-07-08 12:34:11.198333742 +0000 2023-07-08T12:37:06.113 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-07-08 12:34:11.198333742 +0000 2023-07-08T12:37:06.113 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2023-07-08T12:37:06.114 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-08T12:37:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-07-08T12:37:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-07-08T12:37:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000217028 s, 2.4 MB/s 2023-07-08T12:37:06.181 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-08T12:37:06.239 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2023-07-08T12:37:06.298 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-08T12:37:06.298 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-08T12:37:06.298 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 364483 Links: 1 2023-07-08T12:37:06.298 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-08T12:37:06.298 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2023-07-08T12:37:06.298 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-07-08 12:36:55.053781146 +0000 2023-07-08T12:37:06.299 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-07-08 12:34:11.720329843 +0000 2023-07-08T12:37:06.299 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-07-08 12:34:11.720329843 +0000 2023-07-08T12:37:06.299 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2023-07-08T12:37:06.299 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-08T12:37:06.364 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-07-08T12:37:06.364 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-07-08T12:37:06.364 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000198953 s, 2.6 MB/s 2023-07-08T12:37:06.365 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-08T12:37:06.423 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2023-07-08T12:37:06.482 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-08T12:37:06.482 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-08T12:37:06.482 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 380528 Links: 1 2023-07-08T12:37:06.483 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-08T12:37:06.483 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2023-07-08T12:37:06.483 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-07-08 12:36:55.053781146 +0000 2023-07-08T12:37:06.483 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-07-08 12:34:12.262325795 +0000 2023-07-08T12:37:06.483 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-07-08 12:34:12.262325795 +0000 2023-07-08T12:37:06.483 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2023-07-08T12:37:06.484 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-08T12:37:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-07-08T12:37:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-07-08T12:37:06.549 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000213087 s, 2.4 MB/s 2023-07-08T12:37:06.550 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-08T12:37:06.608 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-08T12:37:06.609 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:06.609 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/scratch_devs of=/dev/stdout 2023-07-08T12:37:06.627 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-08T12:37:06.627 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 2023-07-08T12:37:06.685 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-08T12:37:06.686 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-08T12:37:06.686 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 380942 Links: 1 2023-07-08T12:37:06.686 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-08T12:37:06.686 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2023-07-08T12:37:06.686 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2023-07-08 12:36:43.639498797 +0000 2023-07-08T12:37:06.686 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2023-07-08 12:33:51.285350878 +0000 2023-07-08T12:37:06.686 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2023-07-08 12:33:51.285350878 +0000 2023-07-08T12:37:06.686 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2023-07-08T12:37:06.687 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-08T12:37:06.752 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2023-07-08T12:37:06.753 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2023-07-08T12:37:06.753 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000265199 s, 1.9 MB/s 2023-07-08T12:37:06.754 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-08T12:37:06.814 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 2023-07-08T12:37:06.872 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-08T12:37:06.872 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-08T12:37:06.872 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 368515 Links: 1 2023-07-08T12:37:06.873 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-08T12:37:06.873 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2023-07-08T12:37:06.873 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2023-07-08 12:36:43.639498797 +0000 2023-07-08T12:37:06.873 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2023-07-08 12:33:51.801343275 +0000 2023-07-08T12:37:06.873 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2023-07-08 12:33:51.801343275 +0000 2023-07-08T12:37:06.873 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2023-07-08T12:37:06.874 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-08T12:37:06.938 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2023-07-08T12:37:06.938 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2023-07-08T12:37:06.939 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000278428 s, 1.8 MB/s 2023-07-08T12:37:06.939 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-08T12:37:06.997 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 2023-07-08T12:37:07.054 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-08T12:37:07.055 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-08T12:37:07.055 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 382090 Links: 1 2023-07-08T12:37:07.055 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-08T12:37:07.055 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2023-07-08T12:37:07.055 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2023-07-08 12:36:43.639498797 +0000 2023-07-08T12:37:07.055 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2023-07-08 12:33:52.314335716 +0000 2023-07-08T12:37:07.056 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2023-07-08 12:33:52.314335716 +0000 2023-07-08T12:37:07.056 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2023-07-08T12:37:07.056 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-08T12:37:07.119 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2023-07-08T12:37:07.119 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2023-07-08T12:37:07.120 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.00024072 s, 2.1 MB/s 2023-07-08T12:37:07.120 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-08T12:37:07.178 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 2023-07-08T12:37:07.235 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-08T12:37:07.236 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-08T12:37:07.236 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 383209 Links: 1 2023-07-08T12:37:07.237 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-08T12:37:07.237 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2023-07-08T12:37:07.237 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2023-07-08 12:36:43.639498797 +0000 2023-07-08T12:37:07.238 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2023-07-08 12:33:52.825328186 +0000 2023-07-08T12:37:07.238 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2023-07-08 12:33:52.825328186 +0000 2023-07-08T12:37:07.238 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2023-07-08T12:37:07.239 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-08T12:37:07.301 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2023-07-08T12:37:07.301 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2023-07-08T12:37:07.301 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000222657 s, 2.3 MB/s 2023-07-08T12:37:07.302 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-08T12:37:07.360 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-08T12:37:07.361 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi160.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-08T12:37:07.361 INFO:tasks.ceph:Generating config... 2023-07-08T12:37:07.364 INFO:tasks.ceph:[client] client mount timeout = 600 2023-07-08T12:37:07.365 INFO:tasks.ceph:[client] debug client = 20 2023-07-08T12:37:07.365 INFO:tasks.ceph:[client] debug ms = 1 2023-07-08T12:37:07.365 INFO:tasks.ceph:[client] pid file = 2023-07-08T12:37:07.366 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-07-08T12:37:07.366 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-07-08T12:37:07.366 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-07-08T12:37:07.367 INFO:tasks.ceph:[mds] debug mds = 20 2023-07-08T12:37:07.367 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-07-08T12:37:07.367 INFO:tasks.ceph:[mds] debug ms = 1 2023-07-08T12:37:07.367 INFO:tasks.ceph:[mds] mds debug frag = True 2023-07-08T12:37:07.367 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-07-08T12:37:07.368 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-07-08T12:37:07.368 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-07-08T12:37:07.368 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-07-08T12:37:07.368 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-07-08T12:37:07.368 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-07-08T12:37:07.369 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-07-08T12:37:07.369 INFO:tasks.ceph:[mgr] debug ms = 1 2023-07-08T12:37:07.369 INFO:tasks.ceph:[mon] debug mon = 20 2023-07-08T12:37:07.369 INFO:tasks.ceph:[mon] debug ms = 1 2023-07-08T12:37:07.369 INFO:tasks.ceph:[mon] debug paxos = 20 2023-07-08T12:37:07.370 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-07-08T12:37:07.370 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-07-08T12:37:07.370 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-07-08T12:37:07.370 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-07-08T12:37:07.370 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-07-08T12:37:07.370 INFO:tasks.ceph:[osd] debug ms = 1 2023-07-08T12:37:07.371 INFO:tasks.ceph:[osd] debug osd = 20 2023-07-08T12:37:07.371 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-07-08T12:37:07.371 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-07-08T12:37:07.371 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-07-08T12:37:07.371 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-07-08T12:37:07.372 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-07-08T12:37:07.372 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-07-08T12:37:07.372 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-07-08T12:37:07.372 INFO:tasks.ceph:Setting up mon.a... 2023-07-08T12:37:07.373 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-07-08T12:37:07.570 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.keyring 2023-07-08T12:37:07.573 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-07-08T12:37:07.638 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-08T12:37:07.667 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.12'), ('mon.b', '172.21.15.160'), ('mon.c', '[v2:172.21.15.160:3301,v1:172.21.15.160:6790]')] 2023-07-08T12:37:07.668 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.12,172.21.15.160,[v2:172.21.15.160:3301,v1:172.21.15.160:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-07-08T12:37:07.669 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:07.670 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-07-08T12:37:07.724 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.12 --add b 172.21.15.160 --addv c '[v2:172.21.15.160:3301,v1:172.21.15.160:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:07.816 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:07.817 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: generated fsid 71d270f5-58a7-4223-8d07-37436121729a 2023-07-08T12:37:07.817 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = pacific 2023-07-08T12:37:07.817 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 0 2023-07-08T12:37:07.817 INFO:teuthology.orchestra.run.smithi012.stdout:fsid 71d270f5-58a7-4223-8d07-37436121729a 2023-07-08T12:37:07.818 INFO:teuthology.orchestra.run.smithi012.stdout:last_changed 2023-07-08T12:37:07.815136+0000 2023-07-08T12:37:07.818 INFO:teuthology.orchestra.run.smithi012.stdout:created 2023-07-08T12:37:07.815136+0000 2023-07-08T12:37:07.818 INFO:teuthology.orchestra.run.smithi012.stdout:min_mon_release 16 (pacific) 2023-07-08T12:37:07.819 INFO:teuthology.orchestra.run.smithi012.stdout:election_strategy: 1 2023-07-08T12:37:07.819 INFO:teuthology.orchestra.run.smithi012.stdout:0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2023-07-08T12:37:07.819 INFO:teuthology.orchestra.run.smithi012.stdout:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2023-07-08T12:37:07.819 INFO:teuthology.orchestra.run.smithi012.stdout:2: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.c 2023-07-08T12:37:07.820 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-07-08T12:37:07.822 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-07-08T12:37:07.840 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 71d270f5-58a7-4223-8d07-37436121729a... 2023-07-08T12:37:07.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-08T12:37:07.882 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-08T12:37:07.927 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2023-07-08T12:37:07.927 INFO:teuthology.orchestra.run.smithi012.stdout: chdir = "" 2023-07-08T12:37:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-08T12:37:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: auth supported = cephx 2023-07-08T12:37:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: filestore xattr use omap = true 2023-07-08T12:37:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: mon clock drift allowed = 1.000 2023-07-08T12:37:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: osd crush chooseleaf type = 0 2023-07-08T12:37:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: auth debug = true 2023-07-08T12:37:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on old message = true 2023-07-08T12:37:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on bug = true 2023-07-08T12:37:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-08T12:37:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: mon pg warn max object skew = 0 2023-07-08T12:37:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: # disable pg_autoscaler by default for new pools 2023-07-08T12:37:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-08T12:37:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default size = 2 2023-07-08T12:37:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow primary affinity = true 2023-07-08T12:37:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow pg remap = true 2023-07-08T12:37:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on legacy crush tunables = false 2023-07-08T12:37:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on crush straw calc version zero = false 2023-07-08T12:37:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on no sortbitwise = false 2023-07-08T12:37:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on osd down out interval zero = false 2023-07-08T12:37:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on too few osds = false 2023-07-08T12:37:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-08T12:37:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-08T12:37:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: mon_allow_pool_size_one = true 2023-07-08T12:37:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-08T12:37:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: osd default data pool replay window = 5 2023-07-08T12:37:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: mon allow pool delete = true 2023-07-08T12:37:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: mon cluster log file level = debug 2023-07-08T12:37:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: debug asserts on shutdown = true 2023-07-08T12:37:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: mon health detail to clog = false 2023-07-08T12:37:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: mon host = "172.21.15.12,172.21.15.160,[v2:172.21.15.160:3301,v1:172.21.15.160:6790]" 2023-07-08T12:37:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on skipped message = False 2023-07-08T12:37:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = 71d270f5-58a7-4223-8d07-37436121729a 2023-07-08T12:37:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.936 INFO:teuthology.orchestra.run.smithi012.stdout:[osd] 2023-07-08T12:37:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: osd journal size = 100 2023-07-08T12:37:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub load threshold = 5.0 2023-07-08T12:37:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub max interval = 600 2023-07-08T12:37:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: osd mclock profile = high_recovery_ops 2023-07-08T12:37:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: osd recover clone overlap = true 2023-07-08T12:37:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: osd recovery max chunk = 1048576 2023-07-08T12:37:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug shutdown = true 2023-07-08T12:37:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug op order = true 2023-07-08T12:37:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug verify stray on activate = true 2023-07-08T12:37:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: osd open classes on start = true 2023-07-08T12:37:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug pg log writeout = true 2023-07-08T12:37:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: osd deep scrub update digest min age = 30 2023-07-08T12:37:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: osd map max advance = 10 2023-07-08T12:37:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: journal zero on create = true 2023-07-08T12:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: filestore ondisk finisher threads = 3 2023-07-08T12:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: filestore apply finisher threads = 3 2023-07-08T12:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: bdev debug aio = true 2023-07-08T12:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug misdirected ops = true 2023-07-08T12:37:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore block size = 96636764160 2023-07-08T12:37:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore fsck on mount = True 2023-07-08T12:37:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluefs = 20 2023-07-08T12:37:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluestore = 20 2023-07-08T12:37:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-07-08T12:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: debug osd = 20 2023-07-08T12:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: debug rocksdb = 10 2023-07-08T12:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd backfillfull_ratio = 0.85 2023-07-08T12:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd full ratio = 0.9 2023-07-08T12:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd nearfull ratio = 0.8 2023-07-08T12:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: osd failsafe full ratio = 0.95 2023-07-08T12:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: osd objectstore = bluestore 2023-07-08T12:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: osd op complaint time = 180 2023-07-08T12:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr] 2023-07-08T12:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-07-08T12:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: debug mgr = 20 2023-07-08T12:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2023-07-08T12:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2023-07-08T12:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2023-07-08T12:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2023-07-08T12:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: mgr/telemetry/nag = false 2023-07-08T12:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout:[mon] 2023-07-08T12:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-07-08T12:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2023-07-08T12:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: debug paxos = 20 2023-07-08T12:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2023-07-08T12:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: mon data avail warn = 5 2023-07-08T12:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: mon mgr mkfs grace = 240 2023-07-08T12:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2023-07-08T12:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd reporter subtree level = osd 2023-07-08T12:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd prime pg temp = true 2023-07-08T12:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2023-07-08T12:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-08T12:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: auth mon ticket ttl = 660 # 11m 2023-07-08T12:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: auth service ticket ttl = 240 # 4m 2023-07-08T12:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: # don't complain about insecure global_id in the test suite 2023-07-08T12:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-08T12:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-08T12:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: # 1m isn't quite enough 2023-07-08T12:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: mon_down_mkfs_grace = 2m 2023-07-08T12:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_filestore_osds = false 2023-07-08T12:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: mon op complaint time = 120 2023-07-08T12:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout:[client] 2023-07-08T12:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: rgw cache enabled = true 2023-07-08T12:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable ops log = true 2023-07-08T12:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable usage log = true 2023-07-08T12:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-08T12:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-08T12:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: client mount timeout = 600 2023-07-08T12:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: debug client = 20 2023-07-08T12:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-07-08T12:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: pid file = "" 2023-07-08T12:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: rados mon op timeout = 900 2023-07-08T12:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: rados osd op timeout = 900 2023-07-08T12:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.a] 2023-07-08T12:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.b] 2023-07-08T12:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.c] 2023-07-08T12:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a] 2023-07-08T12:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.c] 2023-07-08T12:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b] 2023-07-08T12:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.d] 2023-07-08T12:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout:[mds] 2023-07-08T12:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: debug mds = 20 2023-07-08T12:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: debug mds balancer = 20 2023-07-08T12:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-07-08T12:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: mds debug frag = True 2023-07-08T12:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: mds debug scatterstat = True 2023-07-08T12:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: mds op complaint time = 180 2023-07-08T12:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: mds verify scatter = True 2023-07-08T12:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: osd op complaint time = 180 2023-07-08T12:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: rados mon op timeout = 900 2023-07-08T12:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: rados osd op timeout = 900 2023-07-08T12:37:07.959 INFO:teuthology.orchestra.run.smithi160.stdout:[global] 2023-07-08T12:37:07.959 INFO:teuthology.orchestra.run.smithi160.stdout: chdir = "" 2023-07-08T12:37:07.960 INFO:teuthology.orchestra.run.smithi160.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-08T12:37:07.960 INFO:teuthology.orchestra.run.smithi160.stdout: auth supported = cephx 2023-07-08T12:37:07.960 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.960 INFO:teuthology.orchestra.run.smithi160.stdout: filestore xattr use omap = true 2023-07-08T12:37:07.961 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.961 INFO:teuthology.orchestra.run.smithi160.stdout: mon clock drift allowed = 1.000 2023-07-08T12:37:07.961 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.961 INFO:teuthology.orchestra.run.smithi160.stdout: osd crush chooseleaf type = 0 2023-07-08T12:37:07.961 INFO:teuthology.orchestra.run.smithi160.stdout: auth debug = true 2023-07-08T12:37:07.962 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.962 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on old message = true 2023-07-08T12:37:07.962 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on bug = true 2023-07-08T12:37:07.962 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.962 INFO:teuthology.orchestra.run.smithi160.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-08T12:37:07.962 INFO:teuthology.orchestra.run.smithi160.stdout: mon pg warn max object skew = 0 2023-07-08T12:37:07.963 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.963 INFO:teuthology.orchestra.run.smithi160.stdout: # disable pg_autoscaler by default for new pools 2023-07-08T12:37:07.963 INFO:teuthology.orchestra.run.smithi160.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-08T12:37:07.963 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.963 INFO:teuthology.orchestra.run.smithi160.stdout: osd pool default size = 2 2023-07-08T12:37:07.963 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.964 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd allow primary affinity = true 2023-07-08T12:37:07.964 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd allow pg remap = true 2023-07-08T12:37:07.964 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on legacy crush tunables = false 2023-07-08T12:37:07.964 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on crush straw calc version zero = false 2023-07-08T12:37:07.965 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on no sortbitwise = false 2023-07-08T12:37:07.965 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on osd down out interval zero = false 2023-07-08T12:37:07.965 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on too few osds = false 2023-07-08T12:37:07.965 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-08T12:37:07.965 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-08T12:37:07.965 INFO:teuthology.orchestra.run.smithi160.stdout: mon_allow_pool_size_one = true 2023-07-08T12:37:07.965 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.966 INFO:teuthology.orchestra.run.smithi160.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-08T12:37:07.966 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.966 INFO:teuthology.orchestra.run.smithi160.stdout: osd default data pool replay window = 5 2023-07-08T12:37:07.966 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.966 INFO:teuthology.orchestra.run.smithi160.stdout: mon allow pool delete = true 2023-07-08T12:37:07.966 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.966 INFO:teuthology.orchestra.run.smithi160.stdout: mon cluster log file level = debug 2023-07-08T12:37:07.966 INFO:teuthology.orchestra.run.smithi160.stdout: debug asserts on shutdown = true 2023-07-08T12:37:07.967 INFO:teuthology.orchestra.run.smithi160.stdout: mon health detail to clog = false 2023-07-08T12:37:07.967 INFO:teuthology.orchestra.run.smithi160.stdout: mon host = "172.21.15.12,172.21.15.160,[v2:172.21.15.160:3301,v1:172.21.15.160:6790]" 2023-07-08T12:37:07.967 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on skipped message = False 2023-07-08T12:37:07.967 INFO:teuthology.orchestra.run.smithi160.stdout: fsid = 71d270f5-58a7-4223-8d07-37436121729a 2023-07-08T12:37:07.967 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.967 INFO:teuthology.orchestra.run.smithi160.stdout:[osd] 2023-07-08T12:37:07.967 INFO:teuthology.orchestra.run.smithi160.stdout: osd journal size = 100 2023-07-08T12:37:07.968 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.968 INFO:teuthology.orchestra.run.smithi160.stdout: osd scrub load threshold = 5.0 2023-07-08T12:37:07.968 INFO:teuthology.orchestra.run.smithi160.stdout: osd scrub max interval = 600 2023-07-08T12:37:07.968 INFO:teuthology.orchestra.run.smithi160.stdout: osd mclock profile = high_recovery_ops 2023-07-08T12:37:07.968 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.969 INFO:teuthology.orchestra.run.smithi160.stdout: osd recover clone overlap = true 2023-07-08T12:37:07.969 INFO:teuthology.orchestra.run.smithi160.stdout: osd recovery max chunk = 1048576 2023-07-08T12:37:07.969 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.970 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug shutdown = true 2023-07-08T12:37:07.970 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug op order = true 2023-07-08T12:37:07.970 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug verify stray on activate = true 2023-07-08T12:37:07.970 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.971 INFO:teuthology.orchestra.run.smithi160.stdout: osd open classes on start = true 2023-07-08T12:37:07.971 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug pg log writeout = true 2023-07-08T12:37:07.971 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.971 INFO:teuthology.orchestra.run.smithi160.stdout: osd deep scrub update digest min age = 30 2023-07-08T12:37:07.972 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.972 INFO:teuthology.orchestra.run.smithi160.stdout: osd map max advance = 10 2023-07-08T12:37:07.972 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.973 INFO:teuthology.orchestra.run.smithi160.stdout: journal zero on create = true 2023-07-08T12:37:07.973 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.973 INFO:teuthology.orchestra.run.smithi160.stdout: filestore ondisk finisher threads = 3 2023-07-08T12:37:07.974 INFO:teuthology.orchestra.run.smithi160.stdout: filestore apply finisher threads = 3 2023-07-08T12:37:07.974 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.974 INFO:teuthology.orchestra.run.smithi160.stdout: bdev debug aio = true 2023-07-08T12:37:07.974 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug misdirected ops = true 2023-07-08T12:37:07.975 INFO:teuthology.orchestra.run.smithi160.stdout: bluestore block size = 96636764160 2023-07-08T12:37:07.975 INFO:teuthology.orchestra.run.smithi160.stdout: bluestore fsck on mount = True 2023-07-08T12:37:07.975 INFO:teuthology.orchestra.run.smithi160.stdout: debug bluefs = 20 2023-07-08T12:37:07.975 INFO:teuthology.orchestra.run.smithi160.stdout: debug bluestore = 20 2023-07-08T12:37:07.975 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2023-07-08T12:37:07.975 INFO:teuthology.orchestra.run.smithi160.stdout: debug osd = 20 2023-07-08T12:37:07.975 INFO:teuthology.orchestra.run.smithi160.stdout: debug rocksdb = 10 2023-07-08T12:37:07.976 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd backfillfull_ratio = 0.85 2023-07-08T12:37:07.976 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd full ratio = 0.9 2023-07-08T12:37:07.976 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd nearfull ratio = 0.8 2023-07-08T12:37:07.976 INFO:teuthology.orchestra.run.smithi160.stdout: osd failsafe full ratio = 0.95 2023-07-08T12:37:07.976 INFO:teuthology.orchestra.run.smithi160.stdout: osd objectstore = bluestore 2023-07-08T12:37:07.976 INFO:teuthology.orchestra.run.smithi160.stdout: osd op complaint time = 180 2023-07-08T12:37:07.976 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.977 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr] 2023-07-08T12:37:07.977 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2023-07-08T12:37:07.977 INFO:teuthology.orchestra.run.smithi160.stdout: debug mgr = 20 2023-07-08T12:37:07.977 INFO:teuthology.orchestra.run.smithi160.stdout: debug mon = 20 2023-07-08T12:37:07.977 INFO:teuthology.orchestra.run.smithi160.stdout: debug auth = 20 2023-07-08T12:37:07.977 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min pgs per osd = 4 2023-07-08T12:37:07.977 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min bytes per osd = 10 2023-07-08T12:37:07.977 INFO:teuthology.orchestra.run.smithi160.stdout: mgr/telemetry/nag = false 2023-07-08T12:37:07.978 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.978 INFO:teuthology.orchestra.run.smithi160.stdout:[mon] 2023-07-08T12:37:07.978 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2023-07-08T12:37:07.978 INFO:teuthology.orchestra.run.smithi160.stdout: debug mon = 20 2023-07-08T12:37:07.978 INFO:teuthology.orchestra.run.smithi160.stdout: debug paxos = 20 2023-07-08T12:37:07.978 INFO:teuthology.orchestra.run.smithi160.stdout: debug auth = 20 2023-07-08T12:37:07.978 INFO:teuthology.orchestra.run.smithi160.stdout: mon data avail warn = 5 2023-07-08T12:37:07.978 INFO:teuthology.orchestra.run.smithi160.stdout: mon mgr mkfs grace = 240 2023-07-08T12:37:07.979 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min pgs per osd = 4 2023-07-08T12:37:07.979 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd reporter subtree level = osd 2023-07-08T12:37:07.979 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd prime pg temp = true 2023-07-08T12:37:07.979 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min bytes per osd = 10 2023-07-08T12:37:07.979 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.979 INFO:teuthology.orchestra.run.smithi160.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-08T12:37:07.979 INFO:teuthology.orchestra.run.smithi160.stdout: auth mon ticket ttl = 660 # 11m 2023-07-08T12:37:07.979 INFO:teuthology.orchestra.run.smithi160.stdout: auth service ticket ttl = 240 # 4m 2023-07-08T12:37:07.980 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.980 INFO:teuthology.orchestra.run.smithi160.stdout: # don't complain about insecure global_id in the test suite 2023-07-08T12:37:07.980 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-08T12:37:07.980 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-08T12:37:07.980 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.980 INFO:teuthology.orchestra.run.smithi160.stdout: # 1m isn't quite enough 2023-07-08T12:37:07.980 INFO:teuthology.orchestra.run.smithi160.stdout: mon_down_mkfs_grace = 2m 2023-07-08T12:37:07.981 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.981 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_filestore_osds = false 2023-07-08T12:37:07.981 INFO:teuthology.orchestra.run.smithi160.stdout: mon op complaint time = 120 2023-07-08T12:37:07.981 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-07-08T12:37:07.981 INFO:teuthology.orchestra.run.smithi160.stdout:[client] 2023-07-08T12:37:07.981 INFO:teuthology.orchestra.run.smithi160.stdout: rgw cache enabled = true 2023-07-08T12:37:07.981 INFO:teuthology.orchestra.run.smithi160.stdout: rgw enable ops log = true 2023-07-08T12:37:07.981 INFO:teuthology.orchestra.run.smithi160.stdout: rgw enable usage log = true 2023-07-08T12:37:07.982 INFO:teuthology.orchestra.run.smithi160.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-08T12:37:07.982 INFO:teuthology.orchestra.run.smithi160.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-08T12:37:07.982 INFO:teuthology.orchestra.run.smithi160.stdout: client mount timeout = 600 2023-07-08T12:37:07.982 INFO:teuthology.orchestra.run.smithi160.stdout: debug client = 20 2023-07-08T12:37:07.982 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2023-07-08T12:37:07.982 INFO:teuthology.orchestra.run.smithi160.stdout: pid file = "" 2023-07-08T12:37:07.982 INFO:teuthology.orchestra.run.smithi160.stdout: rados mon op timeout = 900 2023-07-08T12:37:07.982 INFO:teuthology.orchestra.run.smithi160.stdout: rados osd op timeout = 900 2023-07-08T12:37:07.983 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.a] 2023-07-08T12:37:07.983 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.b] 2023-07-08T12:37:07.983 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.c] 2023-07-08T12:37:07.983 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.a] 2023-07-08T12:37:07.983 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.c] 2023-07-08T12:37:07.983 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.b] 2023-07-08T12:37:07.983 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.d] 2023-07-08T12:37:07.983 INFO:teuthology.orchestra.run.smithi160.stdout:[mds] 2023-07-08T12:37:07.984 INFO:teuthology.orchestra.run.smithi160.stdout: debug mds = 20 2023-07-08T12:37:07.984 INFO:teuthology.orchestra.run.smithi160.stdout: debug mds balancer = 20 2023-07-08T12:37:07.984 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2023-07-08T12:37:07.984 INFO:teuthology.orchestra.run.smithi160.stdout: mds debug frag = True 2023-07-08T12:37:07.984 INFO:teuthology.orchestra.run.smithi160.stdout: mds debug scatterstat = True 2023-07-08T12:37:07.984 INFO:teuthology.orchestra.run.smithi160.stdout: mds op complaint time = 180 2023-07-08T12:37:07.984 INFO:teuthology.orchestra.run.smithi160.stdout: mds verify scatter = True 2023-07-08T12:37:07.985 INFO:teuthology.orchestra.run.smithi160.stdout: osd op complaint time = 180 2023-07-08T12:37:07.985 INFO:teuthology.orchestra.run.smithi160.stdout: rados mon op timeout = 900 2023-07-08T12:37:07.985 INFO:teuthology.orchestra.run.smithi160.stdout: rados osd op timeout = 900 2023-07-08T12:37:07.985 INFO:tasks.ceph:Creating admin key on mon.a... 2023-07-08T12:37:07.985 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-07-08T12:37:08.059 INFO:tasks.ceph:Copying monmap to all nodes... 2023-07-08T12:37:08.059 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:08.060 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-07-08T12:37:08.077 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:08.078 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-07-08T12:37:08.134 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi012.front.sepia.ceph.com 2023-07-08T12:37:08.135 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:08.135 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-08T12:37:08.135 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-08T12:37:08.215 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:08.216 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:08.273 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi160.front.sepia.ceph.com 2023-07-08T12:37:08.273 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:08.274 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-08T12:37:08.274 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-08T12:37:08.313 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:08.313 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:08.370 INFO:tasks.ceph:Setting up mon nodes... 2023-07-08T12:37:08.370 INFO:tasks.ceph:Setting up mgr nodes... 2023-07-08T12:37:08.371 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-07-08T12:37:08.430 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-07-08T12:37:08.433 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-07-08T12:37:08.498 INFO:teuthology.orchestra.run.smithi160.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-07-08T12:37:08.501 INFO:tasks.ceph:Setting up mds nodes... 2023-07-08T12:37:08.502 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-07-08T12:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-07-08T12:37:08.562 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-07-08T12:37:08.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-07-08T12:37:08.688 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-07-08T12:37:08.691 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-07-08T12:37:08.719 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-07-08T12:37:08.777 INFO:teuthology.orchestra.run.smithi160.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-07-08T12:37:08.780 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-07-08T12:37:08.807 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-07-08T12:37:08.901 INFO:teuthology.orchestra.run.smithi160.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-07-08T12:37:08.903 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-07-08T12:37:08.929 INFO:tasks.ceph_client:Setting up client nodes... 2023-07-08T12:37:08.929 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-08T12:37:08.977 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-07-08T12:37:08.992 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-07-08T12:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-07-08T12:37:09.051 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-07-08T12:37:09.094 INFO:teuthology.orchestra.run.smithi160.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-07-08T12:37:09.107 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-07-08T12:37:09.189 INFO:teuthology.orchestra.run.smithi160.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-07-08T12:37:09.202 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-07-08T12:37:09.202 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi160.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-08T12:37:09.203 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-07-08T12:37:09.231 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-08T12:37:09.231 INFO:tasks.ceph:role: osd.0 2023-07-08T12:37:09.231 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com 2023-07-08T12:37:09.232 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-08T12:37:09.305 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-08T12:37:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-08T12:37:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-08T12:37:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-08T12:37:09.306 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-08T12:37:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-07-08T12:37:09.306 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-08T12:37:09.307 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-08T12:37:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-08T12:37:09.307 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-08T12:37:09.733 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2023-07-08T12:37:09.735 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-07-08T12:37:09.736 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-07-08T12:37:09.911 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-07-08T12:37:09.945 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-07-08T12:37:10.013 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-08T12:37:10.013 INFO:tasks.ceph:role: osd.1 2023-07-08T12:37:10.013 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com 2023-07-08T12:37:10.014 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-08T12:37:10.087 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-08T12:37:10.087 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-08T12:37:10.087 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-08T12:37:10.088 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-08T12:37:10.088 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-08T12:37:10.088 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-07-08T12:37:10.089 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-08T12:37:10.089 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-08T12:37:10.089 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-08T12:37:10.089 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-08T12:37:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2023-07-08T12:37:10.510 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-07-08T12:37:10.510 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-07-08T12:37:10.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-07-08T12:37:10.647 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-07-08T12:37:10.715 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-08T12:37:10.716 INFO:tasks.ceph:role: osd.2 2023-07-08T12:37:10.716 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com 2023-07-08T12:37:10.716 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-08T12:37:10.790 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-08T12:37:10.790 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-08T12:37:10.790 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-08T12:37:10.790 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-08T12:37:10.790 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-08T12:37:10.791 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-07-08T12:37:10.791 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-08T12:37:10.791 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-08T12:37:10.791 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-08T12:37:10.791 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-08T12:37:11.231 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2023-07-08T12:37:11.232 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-07-08T12:37:11.233 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-07-08T12:37:11.382 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-07-08T12:37:11.415 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-07-08T12:37:11.483 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-08T12:37:11.483 INFO:tasks.ceph:role: osd.3 2023-07-08T12:37:11.483 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi012.front.sepia.ceph.com 2023-07-08T12:37:11.484 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-08T12:37:11.558 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-08T12:37:11.558 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-08T12:37:11.559 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-08T12:37:11.559 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-08T12:37:11.559 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-08T12:37:11.559 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-07-08T12:37:11.559 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-08T12:37:11.559 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-08T12:37:11.559 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-08T12:37:11.560 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-08T12:37:12.001 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2023-07-08T12:37:12.003 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-07-08T12:37:12.004 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-07-08T12:37:12.174 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-07-08T12:37:12.210 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:12.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:12.312+0000 7f2250bcd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-07-08T12:37:12.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:12.312+0000 7f2250bcd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-07-08T12:37:12.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:12.312+0000 7f2250bcd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-08T12:37:12.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:12.312+0000 7f2250bcd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-08T12:37:12.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:12.312+0000 7f2250bcd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-08T12:37:12.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:12.312+0000 7f2250bcd640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-07-08T12:37:15.224 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-07-08T12:37:15.252 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:15.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:15.337+0000 7fbdb26ea640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-07-08T12:37:15.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:15.337+0000 7fbdb26ea640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-07-08T12:37:15.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:15.337+0000 7fbdb26ea640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-08T12:37:15.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:15.337+0000 7fbdb26ea640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-08T12:37:15.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:15.337+0000 7fbdb26ea640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-08T12:37:15.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:15.338+0000 7fbdb26ea640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-07-08T12:37:18.239 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-07-08T12:37:18.267 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:18.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:18.352+0000 7fa4a8f6c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-07-08T12:37:18.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:18.352+0000 7fa4a8f6c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-07-08T12:37:18.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:18.352+0000 7fa4a8f6c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-08T12:37:18.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:18.352+0000 7fa4a8f6c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-08T12:37:18.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:18.352+0000 7fa4a8f6c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-08T12:37:18.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:18.353+0000 7fa4a8f6c640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-07-08T12:37:21.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-07-08T12:37:21.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:21.371+0000 7f6d89e3a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-07-08T12:37:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:21.372+0000 7f6d89e3a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-07-08T12:37:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:21.372+0000 7f6d89e3a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-08T12:37:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:21.372+0000 7f6d89e3a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-08T12:37:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:21.372+0000 7f6d89e3a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-08T12:37:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:21.372+0000 7f6d89e3a640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-07-08T12:37:24.253 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-07-08T12:37:24.287 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-07-08T12:37:24.320 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-08T12:37:24.321 INFO:tasks.ceph:role: osd.4 2023-07-08T12:37:24.321 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi160.front.sepia.ceph.com 2023-07-08T12:37:24.321 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-08T12:37:24.398 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-08T12:37:24.399 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-08T12:37:24.399 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-08T12:37:24.399 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-08T12:37:24.399 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-08T12:37:24.399 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2023-07-08T12:37:24.399 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-08T12:37:24.399 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-08T12:37:24.400 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-08T12:37:24.400 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-08T12:37:24.840 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2023-07-08T12:37:24.842 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2023-07-08T12:37:24.843 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-07-08T12:37:25.039 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-07-08T12:37:25.082 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-07-08T12:37:25.157 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-08T12:37:25.158 INFO:tasks.ceph:role: osd.5 2023-07-08T12:37:25.158 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi160.front.sepia.ceph.com 2023-07-08T12:37:25.159 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-08T12:37:25.234 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-08T12:37:25.234 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-08T12:37:25.234 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-08T12:37:25.234 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-08T12:37:25.235 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-08T12:37:25.235 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2023-07-08T12:37:25.235 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-08T12:37:25.235 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-08T12:37:25.235 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-08T12:37:25.235 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-08T12:37:25.646 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2023-07-08T12:37:25.648 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2023-07-08T12:37:25.649 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-07-08T12:37:25.775 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-07-08T12:37:25.810 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-07-08T12:37:25.878 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-08T12:37:25.878 INFO:tasks.ceph:role: osd.6 2023-07-08T12:37:25.879 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi160.front.sepia.ceph.com 2023-07-08T12:37:25.879 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-08T12:37:25.954 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-08T12:37:25.954 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-08T12:37:25.955 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-08T12:37:25.955 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-08T12:37:25.955 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-08T12:37:25.955 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2023-07-08T12:37:25.955 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-08T12:37:25.955 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-08T12:37:25.955 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-08T12:37:25.956 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-08T12:37:26.399 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2023-07-08T12:37:26.400 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2023-07-08T12:37:26.400 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-07-08T12:37:26.569 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-07-08T12:37:26.602 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-07-08T12:37:26.670 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-08T12:37:26.670 INFO:tasks.ceph:role: osd.7 2023-07-08T12:37:26.670 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi160.front.sepia.ceph.com 2023-07-08T12:37:26.671 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-08T12:37:26.743 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-08T12:37:26.743 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-08T12:37:26.743 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-08T12:37:26.744 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-08T12:37:26.744 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-08T12:37:26.744 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2023-07-08T12:37:26.744 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-08T12:37:26.744 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-08T12:37:26.744 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-08T12:37:26.744 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-08T12:37:27.182 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2023-07-08T12:37:27.183 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2023-07-08T12:37:27.184 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-07-08T12:37:27.371 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-07-08T12:37:27.405 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:27.518 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:27.514+0000 7f55858a2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-07-08T12:37:27.518 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:27.515+0000 7f55858a2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-07-08T12:37:27.518 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:27.515+0000 7f55858a2640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-08T12:37:27.518 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:27.515+0000 7f55858a2640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-08T12:37:27.518 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:27.515+0000 7f55858a2640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-08T12:37:27.519 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:27.515+0000 7f55858a2640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-07-08T12:37:30.383 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-07-08T12:37:30.416 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:30.507 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:30.504+0000 7fe52a960640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-07-08T12:37:30.507 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:30.504+0000 7fe52a960640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-07-08T12:37:30.507 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:30.504+0000 7fe52a960640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-08T12:37:30.507 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:30.504+0000 7fe52a960640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-08T12:37:30.507 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:30.504+0000 7fe52a960640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-08T12:37:30.508 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:30.505+0000 7fe52a960640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-07-08T12:37:33.373 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-07-08T12:37:33.406 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:33.498 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:33.495+0000 7f0e3051d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-07-08T12:37:33.499 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:33.495+0000 7f0e3051d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-07-08T12:37:33.499 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:33.495+0000 7f0e3051d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-08T12:37:33.499 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:33.495+0000 7f0e3051d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-08T12:37:33.500 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:33.495+0000 7f0e3051d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-08T12:37:33.500 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:33.496+0000 7f0e3051d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-07-08T12:37:35.720 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:37:35.723 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:37:36.361 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-07-08T12:37:36.392 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:36.482 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:36.480+0000 7f81fc693640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-07-08T12:37:36.483 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:36.480+0000 7f81fc693640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-07-08T12:37:36.483 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:36.480+0000 7f81fc693640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-08T12:37:36.483 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:36.480+0000 7f81fc693640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-08T12:37:36.483 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:36.480+0000 7f81fc693640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-08T12:37:36.484 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:36.480+0000 7f81fc693640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-07-08T12:37:39.354 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-07-08T12:37:39.385 INFO:tasks.ceph:Reading keys from all nodes... 2023-07-08T12:37:39.386 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:39.386 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-07-08T12:37:39.420 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:39.420 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-07-08T12:37:39.492 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:39.492 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-07-08T12:37:39.563 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:39.564 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-07-08T12:37:39.635 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:39.636 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-07-08T12:37:39.708 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:39.709 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-07-08T12:37:39.779 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:39.779 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-07-08T12:37:39.849 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:39.850 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-07-08T12:37:39.881 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:39.881 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-07-08T12:37:39.951 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:39.952 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-07-08T12:37:40.021 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:40.021 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-07-08T12:37:40.094 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:40.095 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-07-08T12:37:40.162 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:40.162 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-07-08T12:37:40.229 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:40.230 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-07-08T12:37:40.296 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:40.297 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-07-08T12:37:40.317 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:40.317 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-07-08T12:37:40.377 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:40.377 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-07-08T12:37:40.394 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:40.394 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-07-08T12:37:40.450 INFO:tasks.ceph:Adding keys to all mons... 2023-07-08T12:37:40.451 DEBUG:teuthology.orchestra.run.smithi012:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-08T12:37:40.453 DEBUG:teuthology.orchestra.run.smithi160:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-08T12:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr.y] 2023-07-08T12:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB0WKlkaaigGRAAmINnVITHyAQihLVO3fEr2Q== 2023-07-08T12:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a] 2023-07-08T12:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB0WKlk/OpHIRAAPuAogibhBDej8k8fnXpxpA== 2023-07-08T12:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.c] 2023-07-08T12:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB0WKlk2Ej+KBAAGi85adawEIe5Bhyl2VRNyA== 2023-07-08T12:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.0] 2023-07-08T12:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB4WKlkT0erEhAAT6jFqFu8qEqcCV4e1EUuiA== 2023-07-08T12:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.1] 2023-07-08T12:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB7WKlkAiotFBAA4mjsombJttHweEpkU2+dNA== 2023-07-08T12:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.2] 2023-07-08T12:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB+WKlk5OEWFRAAIJIWpT7tetRJCNe1COBOVA== 2023-07-08T12:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.3] 2023-07-08T12:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCBWKlkHcY8FhAAdHT4AbuyWeBD6dYkIoEBOQ== 2023-07-08T12:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr.x] 2023-07-08T12:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB0WKlkHzeqHRAATq0FhzPUkz0sKSEOl27AxQ== 2023-07-08T12:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b] 2023-07-08T12:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB0WKlkTUpCLhAA61h48ZyB7uUwQ/1Bgy91Pg== 2023-07-08T12:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.d] 2023-07-08T12:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB0WKlkM3qlNRAApFyRs7mjHdiBE0yvy4piIA== 2023-07-08T12:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.4] 2023-07-08T12:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCHWKlkJo3IHhAAl8N/CzlnspcQapOVcRjpIQ== 2023-07-08T12:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.5] 2023-07-08T12:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCKWKlkC9ImHhAAV4i9xjI3iPq4VSx01my0AQ== 2023-07-08T12:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.6] 2023-07-08T12:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCNWKlko4ijHRAAIQbuPDCHjpC4KbQayUdinQ== 2023-07-08T12:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.7] 2023-07-08T12:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCQWKlkOPO0HBAAJDU9sOOoFBqX1o8BFizu3w== 2023-07-08T12:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2023-07-08T12:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB0WKlkvys7OhAA7XADn3L9fMd0Gow4lK3U0g== 2023-07-08T12:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout:[client.2] 2023-07-08T12:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB1WKlkSEU1AhAAekux8ymGBJEH4ZjtKKD9mg== 2023-07-08T12:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout:[client.1] 2023-07-08T12:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB1WKlkeYeZBRAAG9I1KVYLBQHMvW/zl1UBqQ== 2023-07-08T12:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout:[client.3] 2023-07-08T12:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB1WKlkdc1BCxAAvYEEUhu/qoH4lZ31bg+D3A== 2023-07-08T12:37:40.515 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr.y] 2023-07-08T12:37:40.515 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB0WKlkaaigGRAAmINnVITHyAQihLVO3fEr2Q== 2023-07-08T12:37:40.516 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.a] 2023-07-08T12:37:40.516 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB0WKlk/OpHIRAAPuAogibhBDej8k8fnXpxpA== 2023-07-08T12:37:40.516 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.c] 2023-07-08T12:37:40.516 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB0WKlk2Ej+KBAAGi85adawEIe5Bhyl2VRNyA== 2023-07-08T12:37:40.516 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.0] 2023-07-08T12:37:40.516 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB4WKlkT0erEhAAT6jFqFu8qEqcCV4e1EUuiA== 2023-07-08T12:37:40.516 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.1] 2023-07-08T12:37:40.517 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB7WKlkAiotFBAA4mjsombJttHweEpkU2+dNA== 2023-07-08T12:37:40.517 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.2] 2023-07-08T12:37:40.517 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB+WKlk5OEWFRAAIJIWpT7tetRJCNe1COBOVA== 2023-07-08T12:37:40.517 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.3] 2023-07-08T12:37:40.517 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQCBWKlkHcY8FhAAdHT4AbuyWeBD6dYkIoEBOQ== 2023-07-08T12:37:40.517 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr.x] 2023-07-08T12:37:40.517 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB0WKlkHzeqHRAATq0FhzPUkz0sKSEOl27AxQ== 2023-07-08T12:37:40.518 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.b] 2023-07-08T12:37:40.518 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB0WKlkTUpCLhAA61h48ZyB7uUwQ/1Bgy91Pg== 2023-07-08T12:37:40.518 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.d] 2023-07-08T12:37:40.518 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB0WKlkM3qlNRAApFyRs7mjHdiBE0yvy4piIA== 2023-07-08T12:37:40.518 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.4] 2023-07-08T12:37:40.519 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQCHWKlkJo3IHhAAl8N/CzlnspcQapOVcRjpIQ== 2023-07-08T12:37:40.519 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.5] 2023-07-08T12:37:40.519 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQCKWKlkC9ImHhAAV4i9xjI3iPq4VSx01my0AQ== 2023-07-08T12:37:40.519 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.6] 2023-07-08T12:37:40.519 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQCNWKlko4ijHRAAIQbuPDCHjpC4KbQayUdinQ== 2023-07-08T12:37:40.519 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.7] 2023-07-08T12:37:40.520 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQCQWKlkOPO0HBAAJDU9sOOoFBqX1o8BFizu3w== 2023-07-08T12:37:40.520 INFO:teuthology.orchestra.run.smithi160.stdout:[client.0] 2023-07-08T12:37:40.520 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB0WKlkvys7OhAA7XADn3L9fMd0Gow4lK3U0g== 2023-07-08T12:37:40.520 INFO:teuthology.orchestra.run.smithi160.stdout:[client.2] 2023-07-08T12:37:40.520 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB1WKlkSEU1AhAAekux8ymGBJEH4ZjtKKD9mg== 2023-07-08T12:37:40.520 INFO:teuthology.orchestra.run.smithi160.stdout:[client.1] 2023-07-08T12:37:40.520 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB1WKlkeYeZBRAAG9I1KVYLBQHMvW/zl1UBqQ== 2023-07-08T12:37:40.520 INFO:teuthology.orchestra.run.smithi160.stdout:[client.3] 2023-07-08T12:37:40.521 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB1WKlkdc1BCxAAvYEEUhu/qoH4lZ31bg+D3A== 2023-07-08T12:37:40.522 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-08T12:37:40.542 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-08T12:37:40.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-08T12:37:40.617 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-08T12:37:40.674 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-08T12:37:40.678 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-08T12:37:40.734 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:40.737 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:40.792 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:40.794 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:40.848 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:40.850 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:40.903 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:40.906 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:40.959 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-08T12:37:40.962 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-08T12:37:41.018 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-08T12:37:41.022 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-08T12:37:41.079 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-08T12:37:41.082 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-08T12:37:41.138 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:41.141 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:41.194 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:41.196 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:41.249 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:41.252 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:41.306 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:41.308 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-08T12:37:41.361 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-08T12:37:41.364 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-08T12:37:41.422 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-08T12:37:41.425 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-08T12:37:41.481 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-08T12:37:41.484 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-08T12:37:41.541 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-08T12:37:41.544 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-08T12:37:41.601 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-07-08T12:37:41.601 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-07-08T12:37:41.629 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-08T12:37:41.728 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-07-08T12:37:41.755 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-07-08T12:37:41.782 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-08T12:37:41.881 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-07-08T12:37:41.908 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-07-08T12:37:41.975 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-08T12:37:42.073 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-07-08T12:37:42.102 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:42.105 DEBUG:teuthology.orchestra.run.smithi160:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-08T12:37:42.159 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-07-08T12:37:42.159 INFO:tasks.ceph.mon.a:Restarting daemon 2023-07-08T12:37:42.160 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-07-08T12:37:42.163 INFO:tasks.ceph.mon.a:Started 2023-07-08T12:37:42.163 INFO:tasks.ceph.mon.b:Restarting daemon 2023-07-08T12:37:42.163 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-07-08T12:37:42.201 INFO:tasks.ceph.mon.b:Started 2023-07-08T12:37:42.201 INFO:tasks.ceph.mon.c:Restarting daemon 2023-07-08T12:37:42.201 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-07-08T12:37:42.204 INFO:tasks.ceph.mon.c:Started 2023-07-08T12:37:42.204 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-07-08T12:37:42.205 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-07-08T12:37:42.205 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-07-08T12:37:42.208 INFO:tasks.ceph.mgr.y:Started 2023-07-08T12:37:42.208 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-07-08T12:37:42.209 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-07-08T12:37:42.210 INFO:tasks.ceph.mgr.x:Started 2023-07-08T12:37:42.211 INFO:tasks.ceph:Setting crush tunables to default 2023-07-08T12:37:42.211 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd crush tunables default 2023-07-08T12:37:42.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.337+0000 7ffa65aec700 1 Processor -- start 2023-07-08T12:37:42.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.337+0000 7ffa65aec700 1 -- start start 2023-07-08T12:37:42.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa65aec700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffa6015a0d0 0x7ffa600d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:42.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa65aec700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa6015d300 0x7ffa601607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:42.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa65aec700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffa600cd360 0x7ffa600cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:42.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa65aec700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa601603a0 con 0x7ffa600cd360 2023-07-08T12:37:42.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa65aec700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa60160000 con 0x7ffa6015a0d0 2023-07-08T12:37:42.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa65aec700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600cdc90 con 0x7ffa6015d300 2023-07-08T12:37:42.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5ffff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffa600cd360 0x7ffa600cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:42.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5ffff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffa600cd360 0x7ffa600cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48454/0 (socket says 172.21.15.12:48454) 2023-07-08T12:37:42.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5ffff700 1 -- 172.21.15.12:0/802486247 learned_addr learned my addr 172.21.15.12:0/802486247 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:37:42.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5ffff700 1 -- 172.21.15.12:0/802486247 >> v2:172.21.15.12:3300/0 conn(0x7ffa600cd360 msgr2=0x7ffa600cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-07-08T12:37:42.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5ffff700 1 -- 172.21.15.12:0/802486247 >> v2:172.21.15.12:3300/0 conn(0x7ffa600cd360 msgr2=0x7ffa600cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-08T12:37:42.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5ffff700 1 --2- 172.21.15.12:0/802486247 >> v2:172.21.15.12:3300/0 conn(0x7ffa600cd360 0x7ffa600cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-08T12:37:42.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5f7fe700 1 --2- 172.21.15.12:0/802486247 >> v2:172.21.15.160:3300/0 conn(0x7ffa6015a0d0 0x7ffa600d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:42.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5ffff700 1 --2- 172.21.15.12:0/802486247 >> v2:172.21.15.12:3300/0 conn(0x7ffa600cd360 0x7ffa600cd750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-08T12:37:42.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5effd700 1 --2- 172.21.15.12:0/802486247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa6015d300 0x7ffa601607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:42.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5f7fe700 1 -- 172.21.15.12:0/802486247 >> v2:172.21.15.160:3300/0 conn(0x7ffa6015a0d0 msgr2=0x7ffa600d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-08T12:37:42.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5f7fe700 1 -- 172.21.15.12:0/802486247 >> v2:172.21.15.160:3300/0 conn(0x7ffa6015a0d0 msgr2=0x7ffa600d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-08T12:37:42.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5f7fe700 1 --2- 172.21.15.12:0/802486247 >> v2:172.21.15.160:3300/0 conn(0x7ffa6015a0d0 0x7ffa600d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-08T12:37:42.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5f7fe700 1 --2- 172.21.15.12:0/802486247 >> v2:172.21.15.160:3300/0 conn(0x7ffa6015a0d0 0x7ffa600d0170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-08T12:37:42.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5effd700 1 -- 172.21.15.12:0/802486247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa6015d300 msgr2=0x7ffa601607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-07-08T12:37:42.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5effd700 1 -- 172.21.15.12:0/802486247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa6015d300 msgr2=0x7ffa601607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-08T12:37:42.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5effd700 1 --2- 172.21.15.12:0/802486247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa6015d300 0x7ffa601607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-08T12:37:42.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.338+0000 7ffa5effd700 1 --2- 172.21.15.12:0/802486247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa6015d300 0x7ffa601607c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-08T12:37:42.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.539+0000 7ffa5ffff700 1 --2- 172.21.15.12:0/802486247 >> v2:172.21.15.12:3300/0 conn(0x7ffa600cd360 0x7ffa600cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:42.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.539+0000 7ffa5f7fe700 1 --2- 172.21.15.12:0/802486247 >> v2:172.21.15.160:3300/0 conn(0x7ffa6015a0d0 0x7ffa600d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:42.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.539+0000 7ffa5effd700 1 --2- 172.21.15.12:0/802486247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa6015d300 0x7ffa601607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:42.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.539+0000 7ffa5ffff700 1 -- 172.21.15.12:0/802486247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa6015d300 msgr2=0x7ffa601607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:37:42.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.539+0000 7ffa5ffff700 1 --2- 172.21.15.12:0/802486247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa6015d300 0x7ffa601607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.539+0000 7ffa5ffff700 1 -- 172.21.15.12:0/802486247 >> v2:172.21.15.160:3300/0 conn(0x7ffa6015a0d0 msgr2=0x7ffa600d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:37:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.539+0000 7ffa5ffff700 1 --2- 172.21.15.12:0/802486247 >> v2:172.21.15.160:3300/0 conn(0x7ffa6015a0d0 0x7ffa600d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.539+0000 7ffa5ffff700 1 -- 172.21.15.12:0/802486247 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa600ce6a0 con 0x7ffa600cd360 2023-07-08T12:37:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.539+0000 7ffa5ffff700 1 --2- 172.21.15.12:0/802486247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa600cd360 0x7ffa600cd750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa600d1aa0 tx=0x7ffa58009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee9971f37db9170 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.539+0000 7ffa5e7fc700 1 -- 172.21.15.12:0/802486247 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58003e30 con 0x7ffa600cd360 2023-07-08T12:37:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.539+0000 7ffa5e7fc700 1 -- 172.21.15.12:0/802486247 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5800f040 con 0x7ffa600cd360 2023-07-08T12:37:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.539+0000 7ffa5e7fc700 1 -- 172.21.15.12:0/802486247 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58007af0 con 0x7ffa600cd360 2023-07-08T12:37:42.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.540+0000 7ffa65aec700 1 -- 172.21.15.12:0/802486247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa600cd360 msgr2=0x7ffa600cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:42.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.540+0000 7ffa65aec700 1 --2- 172.21.15.12:0/802486247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa600cd360 0x7ffa600cd750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa600d1aa0 tx=0x7ffa58009e40 comp rx=0 tx=0).stop 2023-07-08T12:37:42.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.540+0000 7ffa65aec700 1 -- 172.21.15.12:0/802486247 shutdown_connections 2023-07-08T12:37:42.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.540+0000 7ffa65aec700 1 --2- 172.21.15.12:0/802486247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa600cd360 0x7ffa600cd750 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:42.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.540+0000 7ffa65aec700 1 --2- 172.21.15.12:0/802486247 >> v2:172.21.15.160:3300/0 conn(0x7ffa6015a0d0 0x7ffa600d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:42.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.540+0000 7ffa65aec700 1 --2- 172.21.15.12:0/802486247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa6015d300 0x7ffa601607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:42.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.540+0000 7ffa65aec700 1 -- 172.21.15.12:0/802486247 >> 172.21.15.12:0/802486247 conn(0x7ffa6006e6c0 msgr2=0x7ffa6006ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:42.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.540+0000 7ffa65aec700 1 -- 172.21.15.12:0/802486247 shutdown_connections 2023-07-08T12:37:42.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.540+0000 7ffa65aec700 1 -- 172.21.15.12:0/802486247 wait complete. 2023-07-08T12:37:42.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.542+0000 7ffa65aec700 1 Processor -- start 2023-07-08T12:37:42.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.542+0000 7ffa65aec700 1 -- start start 2023-07-08T12:37:42.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa65aec700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa600cd360 0x7ffa6014d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:42.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa65aec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa6015a0d0 0x7ffa6015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:42.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa65aec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa6015d300 0x7ffa601e6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:42.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa65aec700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6014dc40 con 0x7ffa6015a0d0 2023-07-08T12:37:42.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa65aec700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600b1890 con 0x7ffa6015d300 2023-07-08T12:37:42.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa65aec700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600b1b70 con 0x7ffa600cd360 2023-07-08T12:37:42.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa5effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa6015a0d0 0x7ffa6015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:42.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa5effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa6015a0d0 0x7ffa6015b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48484/0 (socket says 172.21.15.12:48484) 2023-07-08T12:37:42.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa5effd700 1 -- 172.21.15.12:0/4101391989 learned_addr learned my addr 172.21.15.12:0/4101391989 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:37:42.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa5f7fe700 1 --2- 172.21.15.12:0/4101391989 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa600cd360 0x7ffa6014d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa5ffff700 1 --2- 172.21.15.12:0/4101391989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa6015d300 0x7ffa601e6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa5effd700 1 -- 172.21.15.12:0/4101391989 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa600cd360 msgr2=0x7ffa6014d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa5effd700 1 --2- 172.21.15.12:0/4101391989 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa600cd360 0x7ffa6014d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa5effd700 1 -- 172.21.15.12:0/4101391989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa6015d300 msgr2=0x7ffa601e6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa5effd700 1 --2- 172.21.15.12:0/4101391989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa6015d300 0x7ffa601e6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa5effd700 1 -- 172.21.15.12:0/4101391989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa580097c0 con 0x7ffa6015a0d0 2023-07-08T12:37:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa5effd700 1 --2- 172.21.15.12:0/4101391989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa6015a0d0 0x7ffa6015b9d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009730 tx=0x7ffa54009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:42.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa47fff700 1 -- 172.21.15.12:0/4101391989 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54014610 con 0x7ffa6015a0d0 2023-07-08T12:37:42.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa47fff700 1 -- 172.21.15.12:0/4101391989 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa54014770 con 0x7ffa6015a0d0 2023-07-08T12:37:42.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa65aec700 1 -- 172.21.15.12:0/4101391989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa600b1e50 con 0x7ffa6015a0d0 2023-07-08T12:37:42.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.544+0000 7ffa47fff700 1 -- 172.21.15.12:0/4101391989 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54014ae0 con 0x7ffa6015a0d0 2023-07-08T12:37:42.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.545+0000 7ffa65aec700 1 -- 172.21.15.12:0/4101391989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa600b2410 con 0x7ffa6015a0d0 2023-07-08T12:37:42.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.545+0000 7ffa47fff700 1 -- 172.21.15.12:0/4101391989 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffa54014c40 con 0x7ffa6015a0d0 2023-07-08T12:37:42.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.545+0000 7ffa47fff700 1 -- 172.21.15.12:0/4101391989 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffa5401c3b0 con 0x7ffa6015a0d0 2023-07-08T12:37:42.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.545+0000 7ffa65aec700 1 -- 172.21.15.12:0/4101391989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa600b2630 con 0x7ffa6015a0d0 2023-07-08T12:37:42.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.548+0000 7ffa47fff700 1 -- 172.21.15.12:0/4101391989 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ffa54018020 con 0x7ffa6015a0d0 2023-07-08T12:37:42.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.627+0000 7ffa65aec700 1 -- 172.21.15.12:0/4101391989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ffa6004ea60 con 0x7ffa6015a0d0 2023-07-08T12:37:42.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.629+0000 7ffa47fff700 1 -- 172.21.15.12:0/4101391989 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7ffa5400f040 con 0x7ffa6015a0d0 2023-07-08T12:37:42.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.630+0000 7ffa65aec700 1 -- 172.21.15.12:0/4101391989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa6015a0d0 msgr2=0x7ffa6015b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:42.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.630+0000 7ffa65aec700 1 --2- 172.21.15.12:0/4101391989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa6015a0d0 0x7ffa6015b9d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009730 tx=0x7ffa54009a40 comp rx=0 tx=0).stop 2023-07-08T12:37:42.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.630+0000 7ffa65aec700 1 -- 172.21.15.12:0/4101391989 shutdown_connections 2023-07-08T12:37:42.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.630+0000 7ffa65aec700 1 --2- 172.21.15.12:0/4101391989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa6015d300 0x7ffa601e6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:42.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.630+0000 7ffa65aec700 1 --2- 172.21.15.12:0/4101391989 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa600cd360 0x7ffa6014d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:42.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.630+0000 7ffa65aec700 1 --2- 172.21.15.12:0/4101391989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa6015a0d0 0x7ffa6015b9d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:42.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.630+0000 7ffa65aec700 1 -- 172.21.15.12:0/4101391989 >> 172.21.15.12:0/4101391989 conn(0x7ffa6006e6c0 msgr2=0x7ffa60158720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:42.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.630+0000 7ffa65aec700 1 -- 172.21.15.12:0/4101391989 shutdown_connections 2023-07-08T12:37:42.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:42.630+0000 7ffa65aec700 1 -- 172.21.15.12:0/4101391989 wait complete. 2023-07-08T12:37:42.634 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2023-07-08T12:37:42.645 INFO:tasks.ceph:check_enable_crimson: False 2023-07-08T12:37:42.645 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-07-08T12:37:42.645 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:42.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-07-08T12:37:42.648 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:42.646+0000 7fc391c1f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-08T12:37:42.651 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:42.649+0000 7f6f35d8c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-08T12:37:42.674 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:42.675 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-07-08T12:37:42.714 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:42.711+0000 7fc391c1f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-08T12:37:42.722 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:42.720+0000 7f6f35d8c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-08T12:37:42.740 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:42.741 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-07-08T12:37:42.789 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:42.786+0000 7fc391c1f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-08T12:37:42.803 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:42.801+0000 7f6f35d8c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-08T12:37:42.808 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:37:42.808 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-07-08T12:37:42.866 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:42.864+0000 7fc391c1f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-08T12:37:42.876 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:42.876 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-07-08T12:37:42.887 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:42.885+0000 7f6f35d8c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-08T12:37:42.902 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:42.903 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-07-08T12:37:42.969 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:42.969 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-07-08T12:37:43.017 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:43.015+0000 7fc391c1f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-08T12:37:43.036 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:37:43.036 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-07-08T12:37:43.038 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:43.036+0000 7f6f35d8c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-08T12:37:43.105 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new f72a321b-c04e-4997-a4a6-3b1feb214dcb 0 2023-07-08T12:37:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.264+0000 7fa22368d700 1 Processor -- start 2023-07-08T12:37:43.267 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.264+0000 7fa22368d700 1 -- start start 2023-07-08T12:37:43.269 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa22368d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa21c0cf3a0 0x7fa21c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.269 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa22368d700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21c160ff0 con 0x7fa21c06a470 2023-07-08T12:37:43.269 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa22368d700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21c0cbcf0 con 0x7fa21c0cb920 2023-07-08T12:37:43.269 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa22368d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c160bd0 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.269 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa220c28700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa21c0cf3a0 0x7fa21c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.269 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa221429700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa21c0cb920 0x7fa21c0cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:42862/0 (socket says 172.21.15.160:42862) 2023-07-08T12:37:43.269 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa221429700 1 -- 172.21.15.160:0/2772366970 learned_addr learned my addr 172.21.15.160:0/2772366970 (peer_addr_for_me v1:172.21.15.160:0/0) 2023-07-08T12:37:43.270 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa213fff700 1 -- 172.21.15.160:0/2772366970 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985123072 0 0) 0x7fa218001110 con 0x7fa21c0cb920 2023-07-08T12:37:43.270 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa213fff700 1 -- 172.21.15.160:0/2772366970 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2080035b0 con 0x7fa21c0cb920 2023-07-08T12:37:43.270 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa220c28700 1 -- 172.21.15.160:0/2772366970 >> v1:172.21.15.160:6789/0 conn(0x7fa21c0cb920 legacy=0x7fa21c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.270 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa220c28700 1 -- 172.21.15.160:0/2772366970 >> v1:172.21.15.12:6789/0 conn(0x7fa21c06a470 legacy=0x7fa21c06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.270 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa220c28700 1 -- 172.21.15.160:0/2772366970 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c071550 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.270 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa220c28700 1 --2- 172.21.15.160:0/2772366970 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa21c0cf3a0 0x7fa21c069f30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009870 tx=0x7fa20c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69cc4be09fe38529 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:43.270 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa213fff700 1 -- 172.21.15.160:0/2772366970 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c011830 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.271 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa213fff700 1 -- 172.21.15.160:0/2772366970 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c011990 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.271 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.265+0000 7fa213fff700 1 -- 172.21.15.160:0/2772366970 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c011d20 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.271 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.266+0000 7fa22368d700 1 -- 172.21.15.160:0/2772366970 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa21c0cf3a0 msgr2=0x7fa21c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.271 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.266+0000 7fa22368d700 1 --2- 172.21.15.160:0/2772366970 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa21c0cf3a0 0x7fa21c069f30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009870 tx=0x7fa20c009b80 comp rx=0 tx=0).stop 2023-07-08T12:37:43.271 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.266+0000 7fa22368d700 1 -- 172.21.15.160:0/2772366970 shutdown_connections 2023-07-08T12:37:43.271 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.266+0000 7fa22368d700 1 --2- 172.21.15.160:0/2772366970 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa21c0cf3a0 0x7fa21c069f30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.271 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.266+0000 7fa22368d700 1 -- 172.21.15.160:0/2772366970 >> 172.21.15.160:0/2772366970 conn(0x7fa21c0cc390 msgr2=0x7fa21c0cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:43.271 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.266+0000 7fa22368d700 1 -- 172.21.15.160:0/2772366970 shutdown_connections 2023-07-08T12:37:43.272 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.266+0000 7fa22368d700 1 -- 172.21.15.160:0/2772366970 wait complete. 2023-07-08T12:37:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.267+0000 7fa22368d700 1 Processor -- start 2023-07-08T12:37:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.267+0000 7fa22368d700 1 -- start start 2023-07-08T12:37:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.268+0000 7fa22368d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa21c06a470 0x7fa21c0ac790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.268+0000 7fa22368d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa21c0cb920 0x7fa21c0c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.268+0000 7fa22368d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa21c0cf3a0 0x7fa21c0b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.268+0000 7fa22368d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0b21c0 con 0x7fa21c0cb920 2023-07-08T12:37:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.268+0000 7fa22368d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0af9b0 con 0x7fa21c06a470 2023-07-08T12:37:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.268+0000 7fa22368d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0ad130 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.268+0000 7fa221c2a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa21c0cf3a0 0x7fa21c0b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.268+0000 7fa220c28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa21c0cb920 0x7fa21c0c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.268+0000 7fa221c2a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa21c0cf3a0 0x7fa21c0b1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:46194/0 (socket says 172.21.15.160:46194) 2023-07-08T12:37:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.268+0000 7fa221c2a700 1 -- 172.21.15.160:0/163393626 learned_addr learned my addr 172.21.15.160:0/163393626 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-07-08T12:37:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.268+0000 7fa221429700 1 --2- 172.21.15.160:0/163393626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa21c06a470 0x7fa21c0ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa221c2a700 1 -- 172.21.15.160:0/163393626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa21c06a470 msgr2=0x7fa21c0ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa221c2a700 1 --2- 172.21.15.160:0/163393626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa21c06a470 0x7fa21c0ac790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa221c2a700 1 -- 172.21.15.160:0/163393626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa21c0cb920 msgr2=0x7fa21c0c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa221c2a700 1 --2- 172.21.15.160:0/163393626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa21c0cb920 0x7fa21c0c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa221c2a700 1 -- 172.21.15.160:0/163393626 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c0094a0 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa220c28700 1 --2- 172.21.15.160:0/163393626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa21c0cb920 0x7fa21c0c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:37:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa221c2a700 1 --2- 172.21.15.160:0/163393626 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa21c0cf3a0 0x7fa21c0b1c80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa21400cb50 tx=0x7fa21400ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa211ffb700 1 -- 172.21.15.160:0/163393626 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214003c00 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa211ffb700 1 -- 172.21.15.160:0/163393626 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa214003d60 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa22368d700 1 -- 172.21.15.160:0/163393626 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c0af3a0 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa211ffb700 1 -- 172.21.15.160:0/163393626 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400aa20 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa211ffb700 1 -- 172.21.15.160:0/163393626 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa2140050d0 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa22368d700 1 -- 172.21.15.160:0/163393626 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c0af640 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.269+0000 7fa211ffb700 1 -- 172.21.15.160:0/163393626 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa2140056c0 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.270+0000 7fa22368d700 1 -- 172.21.15.160:0/163393626 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21c06bfd0 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.273+0000 7fa211ffb700 1 -- 172.21.15.160:0/163393626 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fa214005880 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.359 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.356+0000 7fa22368d700 1 -- 172.21.15.160:0/163393626 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f72a321b-c04e-4997-a4a6-3b1feb214dcb", "id": 0} v 0) v1 -- 0x7fa21c0afb20 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.365 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:43.362+0000 7fc391c1f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-08T12:37:43.365 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.362+0000 7fa211ffb700 1 -- 172.21.15.160:0/163393626 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f72a321b-c04e-4997-a4a6-3b1feb214dcb", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa21402a0b0 con 0x7fa21c0cf3a0 2023-07-08T12:37:43.365 INFO:teuthology.orchestra.run.smithi160.stdout:0 2023-07-08T12:37:43.366 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.363+0000 7fa22368d700 1 -- 172.21.15.160:0/163393626 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa21c0cf3a0 msgr2=0x7fa21c0b1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.366 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.363+0000 7fa22368d700 1 --2- 172.21.15.160:0/163393626 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa21c0cf3a0 0x7fa21c0b1c80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa21400cb50 tx=0x7fa21400ce60 comp rx=0 tx=0).stop 2023-07-08T12:37:43.367 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.363+0000 7fa22368d700 1 -- 172.21.15.160:0/163393626 shutdown_connections 2023-07-08T12:37:43.367 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.363+0000 7fa22368d700 1 --2- 172.21.15.160:0/163393626 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa21c0cf3a0 0x7fa21c0b1c80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.367 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.363+0000 7fa22368d700 1 --2- 172.21.15.160:0/163393626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa21c06a470 0x7fa21c0ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.367 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.363+0000 7fa22368d700 1 --2- 172.21.15.160:0/163393626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa21c0cb920 0x7fa21c0c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.367 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.363+0000 7fa22368d700 1 -- 172.21.15.160:0/163393626 >> 172.21.15.160:0/163393626 conn(0x7fa21c0cc390 msgr2=0x7fa21c06f860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:43.367 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.363+0000 7fa22368d700 1 -- 172.21.15.160:0/163393626 shutdown_connections 2023-07-08T12:37:43.367 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.363+0000 7fa22368d700 1 -- 172.21.15.160:0/163393626 wait complete. 2023-07-08T12:37:43.377 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new 9dacfd32-474b-426e-afd7-0cbeb01f0381 1 2023-07-08T12:37:43.412 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:43.407+0000 7f6f35d8c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-08T12:37:43.447 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:43.444+0000 7fc391c1f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-08T12:37:43.488 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:43.485+0000 7f6f35d8c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-08T12:37:43.502 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.500+0000 7fbf302db700 1 Processor -- start 2023-07-08T12:37:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.500+0000 7fbf302db700 1 -- start start 2023-07-08T12:37:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf302db700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbf281590d0 0x7fbf2815d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf302db700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf280b1aa0 0x7fbf280ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf302db700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf280cd830 con 0x7fbf281594a0 2023-07-08T12:37:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf302db700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf28071550 con 0x7fbf281590d0 2023-07-08T12:37:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf302db700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf28071730 con 0x7fbf280b1aa0 2023-07-08T12:37:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf2e878700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf280b1aa0 0x7fbf280ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf2e878700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf280b1aa0 0x7fbf280ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:46200/0 (socket says 172.21.15.160:46200) 2023-07-08T12:37:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf2e878700 1 -- 172.21.15.160:0/2457845046 learned_addr learned my addr 172.21.15.160:0/2457845046 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-07-08T12:37:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf2e077700 1 --2- 172.21.15.160:0/2457845046 >> v2:172.21.15.12:3300/0 conn(0x7fbf281590d0 0x7fbf2815d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf2e878700 1 -- 172.21.15.160:0/2457845046 >> v2:172.21.15.12:3300/0 conn(0x7fbf281590d0 msgr2=0x7fbf2815d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:37:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf2e878700 1 --2- 172.21.15.160:0/2457845046 >> v2:172.21.15.12:3300/0 conn(0x7fbf281590d0 0x7fbf2815d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf2e878700 1 -- 172.21.15.160:0/2457845046 >> v1:172.21.15.12:6789/0 conn(0x7fbf281594a0 legacy=0x7fbf280b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf2e878700 1 -- 172.21.15.160:0/2457845046 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf28071a10 con 0x7fbf280b1aa0 2023-07-08T12:37:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf2e878700 1 --2- 172.21.15.160:0/2457845046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf280b1aa0 0x7fbf280ccea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009e50 tx=0x7fbf2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b342f8e116dd3ca server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.501+0000 7fbf2d075700 1 -- 172.21.15.160:0/2457845046 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2000e070 con 0x7fbf280b1aa0 2023-07-08T12:37:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.502+0000 7fbf2d075700 1 -- 172.21.15.160:0/2457845046 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2000b910 con 0x7fbf280b1aa0 2023-07-08T12:37:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.502+0000 7fbf2d075700 1 -- 172.21.15.160:0/2457845046 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2000bcc0 con 0x7fbf280b1aa0 2023-07-08T12:37:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.502+0000 7fbf302db700 1 -- 172.21.15.160:0/2457845046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf280b1aa0 msgr2=0x7fbf280ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.502+0000 7fbf302db700 1 --2- 172.21.15.160:0/2457845046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf280b1aa0 0x7fbf280ccea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009e50 tx=0x7fbf2000b330 comp rx=0 tx=0).stop 2023-07-08T12:37:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.502+0000 7fbf302db700 1 -- 172.21.15.160:0/2457845046 shutdown_connections 2023-07-08T12:37:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.502+0000 7fbf302db700 1 --2- 172.21.15.160:0/2457845046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf280b1aa0 0x7fbf280ccea0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.502+0000 7fbf302db700 1 --2- 172.21.15.160:0/2457845046 >> v2:172.21.15.12:3300/0 conn(0x7fbf281590d0 0x7fbf2815d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.502+0000 7fbf302db700 1 -- 172.21.15.160:0/2457845046 >> 172.21.15.160:0/2457845046 conn(0x7fbf280cc390 msgr2=0x7fbf280cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.502+0000 7fbf302db700 1 -- 172.21.15.160:0/2457845046 shutdown_connections 2023-07-08T12:37:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.502+0000 7fbf302db700 1 -- 172.21.15.160:0/2457845046 wait complete. 2023-07-08T12:37:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.504+0000 7fbf302db700 1 Processor -- start 2023-07-08T12:37:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.504+0000 7fbf302db700 1 -- start start 2023-07-08T12:37:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf302db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf280b1aa0 0x7fbf28152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf302db700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf281590d0 0x7fbf281f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf302db700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf281594a0 0x7fbf28157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.514 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf302db700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf281f3cb0 con 0x7fbf280b1aa0 2023-07-08T12:37:43.514 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf302db700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf280c7d90 con 0x7fbf281594a0 2023-07-08T12:37:43.514 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf302db700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf280c9180 con 0x7fbf281590d0 2023-07-08T12:37:43.515 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf2d876700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf281590d0 0x7fbf281f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.515 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf2e878700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf281594a0 0x7fbf28157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.515 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf2d876700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf281590d0 0x7fbf281f3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:46204/0 (socket says 172.21.15.160:46204) 2023-07-08T12:37:43.515 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf2d876700 1 -- 172.21.15.160:0/2909119511 learned_addr learned my addr 172.21.15.160:0/2909119511 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-07-08T12:37:43.516 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf2e077700 1 --2- 172.21.15.160:0/2909119511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf280b1aa0 0x7fbf28152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.516 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf2d876700 1 -- 172.21.15.160:0/2909119511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf281594a0 msgr2=0x7fbf28157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.516 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf2d876700 1 --2- 172.21.15.160:0/2909119511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf281594a0 0x7fbf28157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.516 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf2d876700 1 -- 172.21.15.160:0/2909119511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf280b1aa0 msgr2=0x7fbf28152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.517 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf2d876700 1 --2- 172.21.15.160:0/2909119511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf280b1aa0 0x7fbf28152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.517 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf2d876700 1 -- 172.21.15.160:0/2909119511 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf20009b30 con 0x7fbf281590d0 2023-07-08T12:37:43.517 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf2d876700 1 --2- 172.21.15.160:0/2909119511 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf281590d0 0x7fbf281f3770 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800c4e0 tx=0x7fbf1800c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:43.517 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf1effd700 1 -- 172.21.15.160:0/2909119511 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180052d0 con 0x7fbf281590d0 2023-07-08T12:37:43.518 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf1effd700 1 -- 172.21.15.160:0/2909119511 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf18005430 con 0x7fbf281590d0 2023-07-08T12:37:43.518 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf302db700 1 -- 172.21.15.160:0/2909119511 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf281584e0 con 0x7fbf281590d0 2023-07-08T12:37:43.518 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.505+0000 7fbf1effd700 1 -- 172.21.15.160:0/2909119511 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180038c0 con 0x7fbf281590d0 2023-07-08T12:37:43.519 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.506+0000 7fbf302db700 1 -- 172.21.15.160:0/2909119511 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf280c9690 con 0x7fbf281590d0 2023-07-08T12:37:43.519 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.506+0000 7fbf1effd700 1 -- 172.21.15.160:0/2909119511 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbf18003d20 con 0x7fbf281590d0 2023-07-08T12:37:43.519 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.506+0000 7fbf1effd700 1 -- 172.21.15.160:0/2909119511 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fbf18011e30 con 0x7fbf281590d0 2023-07-08T12:37:43.519 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.507+0000 7fbf302db700 1 -- 172.21.15.160:0/2909119511 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf281f01f0 con 0x7fbf281590d0 2023-07-08T12:37:43.520 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.509+0000 7fbf1effd700 1 -- 172.21.15.160:0/2909119511 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fbf18017070 con 0x7fbf281590d0 2023-07-08T12:37:43.581 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:43.578+0000 7fc391c1f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-08T12:37:43.597 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.594+0000 7fbf302db700 1 -- 172.21.15.160:0/2909119511 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9dacfd32-474b-426e-afd7-0cbeb01f0381", "id": 1} v 0) v1 -- 0x7fbf2804ea60 con 0x7fbf281590d0 2023-07-08T12:37:43.605 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.602+0000 7fbf1effd700 1 -- 172.21.15.160:0/2909119511 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9dacfd32-474b-426e-afd7-0cbeb01f0381", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbf18011590 con 0x7fbf281590d0 2023-07-08T12:37:43.605 INFO:teuthology.orchestra.run.smithi160.stdout:1 2023-07-08T12:37:43.606 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.603+0000 7fbf302db700 1 -- 172.21.15.160:0/2909119511 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf281590d0 msgr2=0x7fbf281f3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.607 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.603+0000 7fbf302db700 1 --2- 172.21.15.160:0/2909119511 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf281590d0 0x7fbf281f3770 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800c4e0 tx=0x7fbf1800c7f0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.607 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.603+0000 7fbf302db700 1 -- 172.21.15.160:0/2909119511 shutdown_connections 2023-07-08T12:37:43.607 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.603+0000 7fbf302db700 1 --2- 172.21.15.160:0/2909119511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf281594a0 0x7fbf28157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.607 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.603+0000 7fbf302db700 1 --2- 172.21.15.160:0/2909119511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf280b1aa0 0x7fbf28152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.607 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.603+0000 7fbf302db700 1 --2- 172.21.15.160:0/2909119511 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf281590d0 0x7fbf281f3770 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.608 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.603+0000 7fbf302db700 1 -- 172.21.15.160:0/2909119511 >> 172.21.15.160:0/2909119511 conn(0x7fbf280cc390 msgr2=0x7fbf280cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:43.608 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.603+0000 7fbf302db700 1 -- 172.21.15.160:0/2909119511 shutdown_connections 2023-07-08T12:37:43.608 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.603+0000 7fbf302db700 1 -- 172.21.15.160:0/2909119511 wait complete. 2023-07-08T12:37:43.618 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new 10b4375f-fca4-4632-b24b-7766da4949ce 2 2023-07-08T12:37:43.625 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:43.623+0000 7f6f35d8c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-08T12:37:43.667 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:43.664+0000 7fc391c1f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-08T12:37:43.710 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:43.707+0000 7f6f35d8c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-08T12:37:43.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.745+0000 7f1a5f864700 1 Processor -- start 2023-07-08T12:37:43.748 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.745+0000 7f1a5f864700 1 -- start start 2023-07-08T12:37:43.749 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5f864700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a580b1aa0 0x7f1a5815d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.749 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5f864700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1a581594a0 0x7f1a580ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5f864700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a580cd830 con 0x7f1a581590d0 2023-07-08T12:37:43.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5f864700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a58071550 con 0x7f1a581594a0 2023-07-08T12:37:43.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5f864700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a58071730 con 0x7f1a580b1aa0 2023-07-08T12:37:43.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5cdff700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a580b1aa0 0x7f1a5815d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5cdff700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a580b1aa0 0x7f1a5815d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:46210/0 (socket says 172.21.15.160:46210) 2023-07-08T12:37:43.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5cdff700 1 -- 172.21.15.160:0/1737879272 learned_addr learned my addr 172.21.15.160:0/1737879272 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-07-08T12:37:43.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5de01700 1 --2- 172.21.15.160:0/1737879272 >> v2:172.21.15.12:3300/0 conn(0x7f1a581594a0 0x7f1a580ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.751 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5cdff700 1 -- 172.21.15.160:0/1737879272 >> v2:172.21.15.12:3300/0 conn(0x7f1a581594a0 msgr2=0x7f1a580ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:37:43.751 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5cdff700 1 --2- 172.21.15.160:0/1737879272 >> v2:172.21.15.12:3300/0 conn(0x7f1a581594a0 0x7f1a580ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.751 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5cdff700 1 -- 172.21.15.160:0/1737879272 >> v1:172.21.15.12:6789/0 conn(0x7f1a581590d0 legacy=0x7f1a580b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.751 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5cdff700 1 -- 172.21.15.160:0/1737879272 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a58071a10 con 0x7f1a580b1aa0 2023-07-08T12:37:43.751 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a5cdff700 1 --2- 172.21.15.160:0/1737879272 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a580b1aa0 0x7f1a5815d810 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a480098f0 tx=0x7f1a48009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7808494d6fe99c45 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:43.751 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a4ffff700 1 -- 172.21.15.160:0/1737879272 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4800b8a0 con 0x7f1a580b1aa0 2023-07-08T12:37:43.751 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a4ffff700 1 -- 172.21.15.160:0/1737879272 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4800ba00 con 0x7f1a580b1aa0 2023-07-08T12:37:43.751 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.746+0000 7f1a4ffff700 1 -- 172.21.15.160:0/1737879272 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4800bd90 con 0x7f1a580b1aa0 2023-07-08T12:37:43.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.747+0000 7f1a5f864700 1 -- 172.21.15.160:0/1737879272 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a580b1aa0 msgr2=0x7f1a5815d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.747+0000 7f1a5f864700 1 --2- 172.21.15.160:0/1737879272 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a580b1aa0 0x7f1a5815d810 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a480098f0 tx=0x7f1a48009c00 comp rx=0 tx=0).stop 2023-07-08T12:37:43.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.747+0000 7f1a5f864700 1 -- 172.21.15.160:0/1737879272 shutdown_connections 2023-07-08T12:37:43.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.747+0000 7f1a5f864700 1 --2- 172.21.15.160:0/1737879272 >> v2:172.21.15.12:3300/0 conn(0x7f1a581594a0 0x7f1a580ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.747+0000 7f1a5f864700 1 --2- 172.21.15.160:0/1737879272 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a580b1aa0 0x7f1a5815d810 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.747+0000 7f1a5f864700 1 -- 172.21.15.160:0/1737879272 >> 172.21.15.160:0/1737879272 conn(0x7f1a580cc390 msgr2=0x7f1a580cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:43.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.747+0000 7f1a5f864700 1 -- 172.21.15.160:0/1737879272 shutdown_connections 2023-07-08T12:37:43.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.747+0000 7f1a5f864700 1 -- 172.21.15.160:0/1737879272 wait complete. 2023-07-08T12:37:43.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.748+0000 7f1a5f864700 1 Processor -- start 2023-07-08T12:37:43.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.748+0000 7f1a5f864700 1 -- start start 2023-07-08T12:37:43.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.749+0000 7f1a5f864700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a580b1aa0 0x7f1a5814aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.756 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.749+0000 7f1a5f864700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a581590d0 0x7f1a580c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.756 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.749+0000 7f1a5f864700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a581594a0 0x7f1a580b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.756 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.749+0000 7f1a5f864700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a580c9830 con 0x7f1a581590d0 2023-07-08T12:37:43.756 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.749+0000 7f1a5f864700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a580c7160 con 0x7f1a580b1aa0 2023-07-08T12:37:43.757 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.749+0000 7f1a5f864700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5814b4c0 con 0x7f1a581594a0 2023-07-08T12:37:43.757 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.749+0000 7f1a5d600700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a580b1aa0 0x7f1a5814aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.757 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.749+0000 7f1a5de01700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a581594a0 0x7f1a580b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.757 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.749+0000 7f1a5d600700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a580b1aa0 0x7f1a5814aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38318/0 (socket says 172.21.15.160:38318) 2023-07-08T12:37:43.758 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.749+0000 7f1a5cdff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a581590d0 0x7f1a580c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.758 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.749+0000 7f1a5d600700 1 -- 172.21.15.160:0/2353819992 learned_addr learned my addr 172.21.15.160:0/2353819992 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-07-08T12:37:43.758 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a5d600700 1 -- 172.21.15.160:0/2353819992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a581594a0 msgr2=0x7f1a580b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.758 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a5d600700 1 --2- 172.21.15.160:0/2353819992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a581594a0 0x7f1a580b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a5d600700 1 -- 172.21.15.160:0/2353819992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a581590d0 msgr2=0x7f1a580c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a5d600700 1 --2- 172.21.15.160:0/2353819992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a581590d0 0x7f1a580c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a5d600700 1 -- 172.21.15.160:0/2353819992 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a48009550 con 0x7f1a580b1aa0 2023-07-08T12:37:43.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a5de01700 1 --2- 172.21.15.160:0/2353819992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a581594a0 0x7f1a580b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:37:43.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a5cdff700 1 --2- 172.21.15.160:0/2353819992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a581590d0 0x7f1a580c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:37:43.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a5d600700 1 --2- 172.21.15.160:0/2353819992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a580b1aa0 0x7f1a5814aa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1a540016c0 tx=0x7f1a540019d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:43.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a4dffb700 1 -- 172.21.15.160:0/2353819992 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54013070 con 0x7f1a580b1aa0 2023-07-08T12:37:43.761 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a4dffb700 1 -- 172.21.15.160:0/2353819992 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5400e2e0 con 0x7f1a580b1aa0 2023-07-08T12:37:43.761 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a5f864700 1 -- 172.21.15.160:0/2353819992 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5814af70 con 0x7f1a580b1aa0 2023-07-08T12:37:43.761 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a4dffb700 1 -- 172.21.15.160:0/2353819992 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a540052f0 con 0x7f1a580b1aa0 2023-07-08T12:37:43.761 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a4dffb700 1 -- 172.21.15.160:0/2353819992 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1a54003980 con 0x7f1a580b1aa0 2023-07-08T12:37:43.762 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a5f864700 1 -- 172.21.15.160:0/2353819992 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a580c6b10 con 0x7f1a580b1aa0 2023-07-08T12:37:43.762 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.750+0000 7f1a4dffb700 1 -- 172.21.15.160:0/2353819992 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f1a5400c450 con 0x7f1a580b1aa0 2023-07-08T12:37:43.763 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.751+0000 7f1a5f864700 1 -- 172.21.15.160:0/2353819992 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5804ea60 con 0x7f1a580b1aa0 2023-07-08T12:37:43.764 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.753+0000 7f1a4dffb700 1 -- 172.21.15.160:0/2353819992 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f1a54005450 con 0x7f1a580b1aa0 2023-07-08T12:37:43.838 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.836+0000 7f1a5f864700 1 -- 172.21.15.160:0/2353819992 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd new", "uuid": "10b4375f-fca4-4632-b24b-7766da4949ce", "id": 2} v 0) v1 -- 0x7f1a580af150 con 0x7f1a580b1aa0 2023-07-08T12:37:43.846 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.843+0000 7f1a4dffb700 1 -- 172.21.15.160:0/2353819992 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "10b4375f-fca4-4632-b24b-7766da4949ce", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1a54003d40 con 0x7f1a580b1aa0 2023-07-08T12:37:43.847 INFO:teuthology.orchestra.run.smithi160.stdout:2 2023-07-08T12:37:43.848 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.844+0000 7f1a5f864700 1 -- 172.21.15.160:0/2353819992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a580b1aa0 msgr2=0x7f1a5814aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.848 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.844+0000 7f1a5f864700 1 --2- 172.21.15.160:0/2353819992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a580b1aa0 0x7f1a5814aa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1a540016c0 tx=0x7f1a540019d0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.848 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.844+0000 7f1a5f864700 1 -- 172.21.15.160:0/2353819992 shutdown_connections 2023-07-08T12:37:43.848 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.844+0000 7f1a5f864700 1 --2- 172.21.15.160:0/2353819992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a581594a0 0x7f1a580b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.848 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.844+0000 7f1a5f864700 1 --2- 172.21.15.160:0/2353819992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a580b1aa0 0x7f1a5814aa30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.848 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.844+0000 7f1a5f864700 1 --2- 172.21.15.160:0/2353819992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a581590d0 0x7f1a580c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.849 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.844+0000 7f1a5f864700 1 -- 172.21.15.160:0/2353819992 >> 172.21.15.160:0/2353819992 conn(0x7f1a580cc390 msgr2=0x7f1a58155220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:43.849 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.845+0000 7f1a5f864700 1 -- 172.21.15.160:0/2353819992 shutdown_connections 2023-07-08T12:37:43.849 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.845+0000 7f1a5f864700 1 -- 172.21.15.160:0/2353819992 wait complete. 2023-07-08T12:37:43.859 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new 98684225-d5af-4e14-9deb-32a335b2f9e2 3 2023-07-08T12:37:43.980 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.977+0000 7f456afe2700 1 Processor -- start 2023-07-08T12:37:43.980 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.977+0000 7f456afe2700 1 -- start start 2023-07-08T12:37:43.983 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f456afe2700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f456415e780 0x7f45640cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.984 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f456afe2700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564071550 0x7f45640719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f456afe2700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456415a700 con 0x7f4564069f30 2023-07-08T12:37:43.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f456afe2700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456415ed20 con 0x7f456415e780 2023-07-08T12:37:43.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f456afe2700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456406f1f0 con 0x7f4564071550 2023-07-08T12:37:43.986 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f4563fff700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f456415e780 0x7f45640cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.986 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f456957f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564071550 0x7f45640719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.986 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f456957f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564071550 0x7f45640719d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:46224/0 (socket says 172.21.15.160:46224) 2023-07-08T12:37:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f4563fff700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f456415e780 0x7f45640cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38326/0 (socket says 172.21.15.160:38326) 2023-07-08T12:37:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f456957f700 1 -- 172.21.15.160:0/374051613 learned_addr learned my addr 172.21.15.160:0/374051613 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-07-08T12:37:43.987 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f4563fff700 1 -- 172.21.15.160:0/374051613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564071550 msgr2=0x7f45640719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:37:43.988 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f4563fff700 1 --2- 172.21.15.160:0/374051613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564071550 0x7f45640719d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.988 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f4563fff700 1 -- 172.21.15.160:0/374051613 >> v1:172.21.15.12:6789/0 conn(0x7f4564069f30 legacy=0x7f456406a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.988 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f4563fff700 1 -- 172.21.15.160:0/374051613 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45640ca6c0 con 0x7f456415e780 2023-07-08T12:37:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f456957f700 1 --2- 172.21.15.160:0/374051613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564071550 0x7f45640719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:37:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f4563fff700 1 --2- 172.21.15.160:0/374051613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f456415e780 0x7f45640cbaf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4550009840 tx=0x7f4550009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96aff9b49ba7f1c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.978+0000 7f45637fe700 1 -- 172.21.15.160:0/374051613 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550011830 con 0x7f456415e780 2023-07-08T12:37:43.989 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.979+0000 7f45637fe700 1 -- 172.21.15.160:0/374051613 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4550011990 con 0x7f456415e780 2023-07-08T12:37:43.990 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.979+0000 7f45637fe700 1 -- 172.21.15.160:0/374051613 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550011d20 con 0x7f456415e780 2023-07-08T12:37:43.990 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.979+0000 7f456afe2700 1 -- 172.21.15.160:0/374051613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f456415e780 msgr2=0x7f45640cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.990 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.979+0000 7f456afe2700 1 --2- 172.21.15.160:0/374051613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f456415e780 0x7f45640cbaf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4550009840 tx=0x7f4550009b50 comp rx=0 tx=0).stop 2023-07-08T12:37:43.990 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.979+0000 7f456afe2700 1 -- 172.21.15.160:0/374051613 shutdown_connections 2023-07-08T12:37:43.990 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.979+0000 7f456afe2700 1 --2- 172.21.15.160:0/374051613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564071550 0x7f45640719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.991 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.979+0000 7f456afe2700 1 --2- 172.21.15.160:0/374051613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f456415e780 0x7f45640cbaf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.991 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.979+0000 7f456afe2700 1 -- 172.21.15.160:0/374051613 >> 172.21.15.160:0/374051613 conn(0x7f45640cc390 msgr2=0x7f45640cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:43.991 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.979+0000 7f456afe2700 1 -- 172.21.15.160:0/374051613 shutdown_connections 2023-07-08T12:37:43.991 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.979+0000 7f456afe2700 1 -- 172.21.15.160:0/374051613 wait complete. 2023-07-08T12:37:43.991 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.981+0000 7f456afe2700 1 Processor -- start 2023-07-08T12:37:43.992 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.981+0000 7f456afe2700 1 -- start start 2023-07-08T12:37:43.994 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f456afe2700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564069f30 0x7f4564153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.994 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f456afe2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4564071550 0x7f45640b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.994 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f456afe2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f456415e780 0x7f45640c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:43.994 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f456afe2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45640b17a0 con 0x7f4564071550 2023-07-08T12:37:43.995 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f456afe2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4564153740 con 0x7f456415e780 2023-07-08T12:37:43.995 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f456afe2700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45640b0b90 con 0x7f4564069f30 2023-07-08T12:37:43.995 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f456957f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f456415e780 0x7f45640c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.995 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f4568d7e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564069f30 0x7f4564153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.996 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f456957f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f456415e780 0x7f45640c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38334/0 (socket says 172.21.15.160:38334) 2023-07-08T12:37:43.996 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f456957f700 1 -- 172.21.15.160:0/3727103236 learned_addr learned my addr 172.21.15.160:0/3727103236 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-07-08T12:37:43.996 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f4568d7e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564069f30 0x7f4564153200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:46228/0 (socket says 172.21.15.160:46228) 2023-07-08T12:37:43.996 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f4563fff700 1 --2- 172.21.15.160:0/3727103236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4564071550 0x7f45640b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:43.997 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f4568d7e700 1 -- 172.21.15.160:0/3727103236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f456415e780 msgr2=0x7f45640c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.997 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f4568d7e700 1 --2- 172.21.15.160:0/3727103236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f456415e780 0x7f45640c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.997 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f4568d7e700 1 -- 172.21.15.160:0/3727103236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4564071550 msgr2=0x7f45640b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:43.997 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f4568d7e700 1 --2- 172.21.15.160:0/3727103236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4564071550 0x7f45640b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:43.998 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f4568d7e700 1 -- 172.21.15.160:0/3727103236 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c009b30 con 0x7f4564069f30 2023-07-08T12:37:43.998 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.982+0000 7f4568d7e700 1 --2- 172.21.15.160:0/3727103236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564069f30 0x7f4564153200 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4558001760 tx=0x7f4558001a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:43.998 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.983+0000 7f45617fa700 1 -- 172.21.15.160:0/3727103236 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558013070 con 0x7f4564069f30 2023-07-08T12:37:43.999 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.983+0000 7f45617fa700 1 -- 172.21.15.160:0/3727103236 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455800e2e0 con 0x7f4564069f30 2023-07-08T12:37:43.999 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.983+0000 7f456afe2700 1 -- 172.21.15.160:0/3727103236 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45500094a0 con 0x7f4564069f30 2023-07-08T12:37:43.999 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.983+0000 7f45617fa700 1 -- 172.21.15.160:0/3727103236 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45580052f0 con 0x7f4564069f30 2023-07-08T12:37:44.000 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.983+0000 7f456afe2700 1 -- 172.21.15.160:0/3727103236 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45640b1150 con 0x7f4564069f30 2023-07-08T12:37:44.000 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.983+0000 7f45617fa700 1 -- 172.21.15.160:0/3727103236 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4558003980 con 0x7f4564069f30 2023-07-08T12:37:44.000 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.983+0000 7f45617fa700 1 -- 172.21.15.160:0/3727103236 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f455800ccf0 con 0x7f4564069f30 2023-07-08T12:37:44.000 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.983+0000 7f456afe2700 1 -- 172.21.15.160:0/3727103236 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45640ca0c0 con 0x7f4564069f30 2023-07-08T12:37:44.001 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:43.986+0000 7f45617fa700 1 -- 172.21.15.160:0/3727103236 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f456415a700 con 0x7f4564069f30 2023-07-08T12:37:44.070 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.068+0000 7f456afe2700 1 -- 172.21.15.160:0/3727103236 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd new", "uuid": "98684225-d5af-4e14-9deb-32a335b2f9e2", "id": 3} v 0) v1 -- 0x7f45640ca390 con 0x7f4564069f30 2023-07-08T12:37:44.078 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.076+0000 7f45617fa700 1 -- 172.21.15.160:0/3727103236 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "98684225-d5af-4e14-9deb-32a335b2f9e2", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f455800c450 con 0x7f4564069f30 2023-07-08T12:37:44.079 INFO:teuthology.orchestra.run.smithi160.stdout:3 2023-07-08T12:37:44.079 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.076+0000 7f456afe2700 1 -- 172.21.15.160:0/3727103236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564069f30 msgr2=0x7f4564153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.080 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.077+0000 7f456afe2700 1 --2- 172.21.15.160:0/3727103236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564069f30 0x7f4564153200 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4558001760 tx=0x7f4558001a70 comp rx=0 tx=0).stop 2023-07-08T12:37:44.080 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.077+0000 7f456afe2700 1 -- 172.21.15.160:0/3727103236 shutdown_connections 2023-07-08T12:37:44.080 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.077+0000 7f456afe2700 1 --2- 172.21.15.160:0/3727103236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f456415e780 0x7f45640c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.080 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.077+0000 7f456afe2700 1 --2- 172.21.15.160:0/3727103236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4564069f30 0x7f4564153200 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.080 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.077+0000 7f456afe2700 1 --2- 172.21.15.160:0/3727103236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4564071550 0x7f45640b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.081 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.077+0000 7f456afe2700 1 -- 172.21.15.160:0/3727103236 >> 172.21.15.160:0/3727103236 conn(0x7f45640cc390 msgr2=0x7f4564159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:44.081 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.077+0000 7f456afe2700 1 -- 172.21.15.160:0/3727103236 shutdown_connections 2023-07-08T12:37:44.081 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.077+0000 7f456afe2700 1 -- 172.21.15.160:0/3727103236 wait complete. 2023-07-08T12:37:44.091 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new d059f27b-b32d-4f7b-a2f1-18b71a396093 4 2023-07-08T12:37:44.136 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:44.131+0000 7fc391c1f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-08T12:37:44.157 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:44.155+0000 7f6f35d8c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-08T12:37:44.214 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.211+0000 7f70e0f73700 1 Processor -- start 2023-07-08T12:37:44.214 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.211+0000 7f70e0f73700 1 -- start start 2023-07-08T12:37:44.217 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.212+0000 7f70e0f73700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc069f30 0x7f70dc0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.217 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.212+0000 7f70e0f73700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70dc15e780 0x7f70dc06a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.217 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.212+0000 7f70e0f73700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70dc15a700 con 0x7f70dc071550 2023-07-08T12:37:44.217 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.212+0000 7f70e0f73700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc15ed20 con 0x7f70dc15e780 2023-07-08T12:37:44.218 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.212+0000 7f70e0f73700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc06f1f0 con 0x7f70dc069f30 2023-07-08T12:37:44.218 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.212+0000 7f70da59c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc069f30 0x7f70dc0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.218 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.212+0000 7f70dad9d700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f70dc071550 0x7f70dc06f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:42910/0 (socket says 172.21.15.160:42910) 2023-07-08T12:37:44.218 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.213+0000 7f70dad9d700 1 -- 172.21.15.160:0/1813720771 learned_addr learned my addr 172.21.15.160:0/1813720771 (peer_addr_for_me v1:172.21.15.160:0/0) 2023-07-08T12:37:44.218 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.213+0000 7f70d9d9b700 1 --2- 172.21.15.160:0/1813720771 >> v2:172.21.15.12:3300/0 conn(0x7f70dc15e780 0x7f70dc06a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.218 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.213+0000 7f70d959a700 1 -- 172.21.15.160:0/1813720771 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947862516 0 0) 0x7f70d4001490 con 0x7f70dc071550 2023-07-08T12:37:44.218 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.213+0000 7f70d959a700 1 -- 172.21.15.160:0/1813720771 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70c40035b0 con 0x7f70dc071550 2023-07-08T12:37:44.218 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.213+0000 7f70da59c700 1 -- 172.21.15.160:0/1813720771 >> v2:172.21.15.12:3300/0 conn(0x7f70dc15e780 msgr2=0x7f70dc06a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:37:44.219 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.213+0000 7f70da59c700 1 --2- 172.21.15.160:0/1813720771 >> v2:172.21.15.12:3300/0 conn(0x7f70dc15e780 0x7f70dc06a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.219 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.213+0000 7f70da59c700 1 -- 172.21.15.160:0/1813720771 >> v1:172.21.15.160:6789/0 conn(0x7f70dc071550 legacy=0x7f70dc06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.219 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.213+0000 7f70da59c700 1 -- 172.21.15.160:0/1813720771 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0ca6c0 con 0x7f70dc069f30 2023-07-08T12:37:44.219 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.213+0000 7f70da59c700 1 --2- 172.21.15.160:0/1813720771 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc069f30 0x7f70dc0cbaf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009840 tx=0x7f70d0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7d8ad5185569e6c server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:44.219 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.213+0000 7f70d959a700 1 -- 172.21.15.160:0/1813720771 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0011830 con 0x7f70dc069f30 2023-07-08T12:37:44.219 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.213+0000 7f70d959a700 1 -- 172.21.15.160:0/1813720771 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d0011990 con 0x7f70dc069f30 2023-07-08T12:37:44.219 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.213+0000 7f70d959a700 1 -- 172.21.15.160:0/1813720771 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0011d20 con 0x7f70dc069f30 2023-07-08T12:37:44.220 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.214+0000 7f70e0f73700 1 -- 172.21.15.160:0/1813720771 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc069f30 msgr2=0x7f70dc0cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.220 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.214+0000 7f70e0f73700 1 --2- 172.21.15.160:0/1813720771 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc069f30 0x7f70dc0cbaf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009840 tx=0x7f70d0009b50 comp rx=0 tx=0).stop 2023-07-08T12:37:44.221 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.214+0000 7f70e0f73700 1 -- 172.21.15.160:0/1813720771 shutdown_connections 2023-07-08T12:37:44.221 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.214+0000 7f70e0f73700 1 --2- 172.21.15.160:0/1813720771 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc069f30 0x7f70dc0cbaf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.221 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.214+0000 7f70e0f73700 1 --2- 172.21.15.160:0/1813720771 >> v2:172.21.15.12:3300/0 conn(0x7f70dc15e780 0x7f70dc06a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.221 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.214+0000 7f70e0f73700 1 -- 172.21.15.160:0/1813720771 >> 172.21.15.160:0/1813720771 conn(0x7f70dc0cc390 msgr2=0x7f70dc0cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:44.221 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.214+0000 7f70e0f73700 1 -- 172.21.15.160:0/1813720771 shutdown_connections 2023-07-08T12:37:44.222 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.214+0000 7f70e0f73700 1 -- 172.21.15.160:0/1813720771 wait complete. 2023-07-08T12:37:44.226 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.215+0000 7f70e0f73700 1 Processor -- start 2023-07-08T12:37:44.226 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.216+0000 7f70e0f73700 1 -- start start 2023-07-08T12:37:44.226 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.216+0000 7f70e0f73700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc069f30 0x7f70dc147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.227 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.216+0000 7f70e0f73700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70dc071550 0x7f70dc1ece70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.227 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.216+0000 7f70e0f73700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70dc15e780 0x7f70dc0c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.227 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.216+0000 7f70e0f73700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0c4380 con 0x7f70dc15e780 2023-07-08T12:37:44.228 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.216+0000 7f70e0f73700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0c7bd0 con 0x7f70dc071550 2023-07-08T12:37:44.228 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.216+0000 7f70e0f73700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0c7eb0 con 0x7f70dc069f30 2023-07-08T12:37:44.228 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.216+0000 7f70da59c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc069f30 0x7f70dc147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.228 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.216+0000 7f70da59c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc069f30 0x7f70dc147540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:46238/0 (socket says 172.21.15.160:46238) 2023-07-08T12:37:44.229 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.216+0000 7f70da59c700 1 -- 172.21.15.160:0/2456717613 learned_addr learned my addr 172.21.15.160:0/2456717613 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-07-08T12:37:44.229 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.216+0000 7f70d9d9b700 1 --2- 172.21.15.160:0/2456717613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70dc071550 0x7f70dc1ece70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.229 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70dad9d700 1 --2- 172.21.15.160:0/2456717613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70dc15e780 0x7f70dc0c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.229 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70da59c700 1 -- 172.21.15.160:0/2456717613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70dc071550 msgr2=0x7f70dc1ece70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.230 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70da59c700 1 --2- 172.21.15.160:0/2456717613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70dc071550 0x7f70dc1ece70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.230 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70da59c700 1 -- 172.21.15.160:0/2456717613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70dc15e780 msgr2=0x7f70dc0c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.231 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70da59c700 1 --2- 172.21.15.160:0/2456717613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70dc15e780 0x7f70dc0c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.231 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70da59c700 1 -- 172.21.15.160:0/2456717613 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d00094a0 con 0x7f70dc069f30 2023-07-08T12:37:44.232 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70d9d9b700 1 --2- 172.21.15.160:0/2456717613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70dc071550 0x7f70dc1ece70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:37:44.232 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70da59c700 1 --2- 172.21.15.160:0/2456717613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc069f30 0x7f70dc147540 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009fd0 tx=0x7f70d0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:44.232 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70caffd700 1 -- 172.21.15.160:0/2456717613 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0004590 con 0x7f70dc069f30 2023-07-08T12:37:44.232 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70caffd700 1 -- 172.21.15.160:0/2456717613 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d001e070 con 0x7f70dc069f30 2023-07-08T12:37:44.232 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70caffd700 1 -- 172.21.15.160:0/2456717613 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d00038e0 con 0x7f70dc069f30 2023-07-08T12:37:44.232 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70e0f73700 1 -- 172.21.15.160:0/2456717613 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70dc0c8190 con 0x7f70dc069f30 2023-07-08T12:37:44.232 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70e0f73700 1 -- 172.21.15.160:0/2456717613 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70dc147a80 con 0x7f70dc069f30 2023-07-08T12:37:44.233 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70caffd700 1 -- 172.21.15.160:0/2456717613 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f70d0003d40 con 0x7f70dc069f30 2023-07-08T12:37:44.233 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.217+0000 7f70caffd700 1 -- 172.21.15.160:0/2456717613 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f70d001a930 con 0x7f70dc069f30 2023-07-08T12:37:44.233 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.218+0000 7f70e0f73700 1 -- 172.21.15.160:0/2456717613 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70dc04ea60 con 0x7f70dc069f30 2023-07-08T12:37:44.233 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.220+0000 7f70caffd700 1 -- 172.21.15.160:0/2456717613 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f70d0020070 con 0x7f70dc069f30 2023-07-08T12:37:44.307 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.304+0000 7f70e0f73700 1 -- 172.21.15.160:0/2456717613 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d059f27b-b32d-4f7b-a2f1-18b71a396093", "id": 4} v 0) v1 -- 0x7f70dc0c7ff0 con 0x7f70dc069f30 2023-07-08T12:37:44.315 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.312+0000 7f70caffd700 1 -- 172.21.15.160:0/2456717613 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d059f27b-b32d-4f7b-a2f1-18b71a396093", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f70dc0c7ff0 con 0x7f70dc069f30 2023-07-08T12:37:44.316 INFO:teuthology.orchestra.run.smithi160.stdout:4 2023-07-08T12:37:44.317 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.313+0000 7f70e0f73700 1 -- 172.21.15.160:0/2456717613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc069f30 msgr2=0x7f70dc147540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.317 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.313+0000 7f70e0f73700 1 --2- 172.21.15.160:0/2456717613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc069f30 0x7f70dc147540 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009fd0 tx=0x7f70d0004750 comp rx=0 tx=0).stop 2023-07-08T12:37:44.317 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.313+0000 7f70e0f73700 1 -- 172.21.15.160:0/2456717613 shutdown_connections 2023-07-08T12:37:44.317 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.313+0000 7f70e0f73700 1 --2- 172.21.15.160:0/2456717613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70dc15e780 0x7f70dc0c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.317 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.313+0000 7f70e0f73700 1 --2- 172.21.15.160:0/2456717613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc069f30 0x7f70dc147540 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.317 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.313+0000 7f70e0f73700 1 --2- 172.21.15.160:0/2456717613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70dc071550 0x7f70dc1ece70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.318 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.313+0000 7f70e0f73700 1 -- 172.21.15.160:0/2456717613 >> 172.21.15.160:0/2456717613 conn(0x7f70dc0cc390 msgr2=0x7f70dc0b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:44.318 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.313+0000 7f70e0f73700 1 -- 172.21.15.160:0/2456717613 shutdown_connections 2023-07-08T12:37:44.318 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.313+0000 7f70e0f73700 1 -- 172.21.15.160:0/2456717613 wait complete. 2023-07-08T12:37:44.327 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new 92528dc3-7627-4bad-9575-9ceda9430855 5 2023-07-08T12:37:44.490 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.487+0000 7fbcaea6d700 1 Processor -- start 2023-07-08T12:37:44.490 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.487+0000 7fbcaea6d700 1 -- start start 2023-07-08T12:37:44.492 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbcaea6d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbca80b4d20 0x7fbca80cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.492 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbcaea6d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbca80d25c0 0x7fbca80d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.493 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbcaea6d700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca8160a00 con 0x7fbca80b8260 2023-07-08T12:37:44.493 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbcaea6d700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca8160ce0 con 0x7fbca80d25c0 2023-07-08T12:37:44.493 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbcaea6d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80cd280 con 0x7fbca80b4d20 2023-07-08T12:37:44.493 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbcac809700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fbca80b8260 0x7fbca80b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:42922/0 (socket says 172.21.15.160:42922) 2023-07-08T12:37:44.494 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbca7fff700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbca80b4d20 0x7fbca80cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.494 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbcac809700 1 -- 172.21.15.160:0/3457562199 learned_addr learned my addr 172.21.15.160:0/3457562199 (peer_addr_for_me v1:172.21.15.160:0/0) 2023-07-08T12:37:44.494 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbcad00a700 1 --2- 172.21.15.160:0/3457562199 >> v2:172.21.15.12:3300/0 conn(0x7fbca80d25c0 0x7fbca80d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.494 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbca77fe700 1 -- 172.21.15.160:0/3457562199 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060824728 0 0) 0x7fbca8160a00 con 0x7fbca80b8260 2023-07-08T12:37:44.494 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbca77fe700 1 -- 172.21.15.160:0/3457562199 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc980035b0 con 0x7fbca80b8260 2023-07-08T12:37:44.494 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbca7fff700 1 -- 172.21.15.160:0/3457562199 >> v2:172.21.15.12:3300/0 conn(0x7fbca80d25c0 msgr2=0x7fbca80d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:37:44.495 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbca7fff700 1 --2- 172.21.15.160:0/3457562199 >> v2:172.21.15.12:3300/0 conn(0x7fbca80d25c0 0x7fbca80d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.495 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbca7fff700 1 -- 172.21.15.160:0/3457562199 >> v1:172.21.15.160:6789/0 conn(0x7fbca80b8260 legacy=0x7fbca80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.495 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.488+0000 7fbca7fff700 1 -- 172.21.15.160:0/3457562199 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca80cd560 con 0x7fbca80b4d20 2023-07-08T12:37:44.495 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.489+0000 7fbca7fff700 1 --2- 172.21.15.160:0/3457562199 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbca80b4d20 0x7fbca80cede0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009840 tx=0x7fbc94009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e457fae206d3e7b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:44.495 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.489+0000 7fbca77fe700 1 -- 172.21.15.160:0/3457562199 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94011830 con 0x7fbca80b4d20 2023-07-08T12:37:44.495 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.489+0000 7fbca77fe700 1 -- 172.21.15.160:0/3457562199 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc94011990 con 0x7fbca80b4d20 2023-07-08T12:37:44.496 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.489+0000 7fbca77fe700 1 -- 172.21.15.160:0/3457562199 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94011d00 con 0x7fbca80b4d20 2023-07-08T12:37:44.496 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.489+0000 7fbcaea6d700 1 -- 172.21.15.160:0/3457562199 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbca80b4d20 msgr2=0x7fbca80cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.496 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.489+0000 7fbcaea6d700 1 --2- 172.21.15.160:0/3457562199 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbca80b4d20 0x7fbca80cede0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009840 tx=0x7fbc94009b50 comp rx=0 tx=0).stop 2023-07-08T12:37:44.496 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.489+0000 7fbcaea6d700 1 -- 172.21.15.160:0/3457562199 shutdown_connections 2023-07-08T12:37:44.496 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.489+0000 7fbcaea6d700 1 --2- 172.21.15.160:0/3457562199 >> v2:172.21.15.12:3300/0 conn(0x7fbca80d25c0 0x7fbca80d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.496 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.489+0000 7fbcaea6d700 1 --2- 172.21.15.160:0/3457562199 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbca80b4d20 0x7fbca80cede0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.497 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.489+0000 7fbcaea6d700 1 -- 172.21.15.160:0/3457562199 >> 172.21.15.160:0/3457562199 conn(0x7fbca80722b0 msgr2=0x7fbca80cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:44.497 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.489+0000 7fbcaea6d700 1 -- 172.21.15.160:0/3457562199 shutdown_connections 2023-07-08T12:37:44.497 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.490+0000 7fbcaea6d700 1 -- 172.21.15.160:0/3457562199 wait complete. 2023-07-08T12:37:44.499 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.491+0000 7fbcaea6d700 1 Processor -- start 2023-07-08T12:37:44.499 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.491+0000 7fbcaea6d700 1 -- start start 2023-07-08T12:37:44.500 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.492+0000 7fbcaea6d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca80b8260 0x7fbca815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.500 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.492+0000 7fbcaea6d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbca80d25c0 0x7fbca81f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.500 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.492+0000 7fbcaea6d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca8153030 0x7fbca815edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.500 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.492+0000 7fbcaea6d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca815f9e0 con 0x7fbca80b8260 2023-07-08T12:37:44.500 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.492+0000 7fbcaea6d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca81f2f00 con 0x7fbca8153030 2023-07-08T12:37:44.500 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.492+0000 7fbcaea6d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca815e020 con 0x7fbca80d25c0 2023-07-08T12:37:44.500 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.492+0000 7fbca7fff700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbca80d25c0 0x7fbca81f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.501 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.492+0000 7fbcad00a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca8153030 0x7fbca815edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.501 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.492+0000 7fbca7fff700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbca80d25c0 0x7fbca81f29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:46250/0 (socket says 172.21.15.160:46250) 2023-07-08T12:37:44.501 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbca7fff700 1 -- 172.21.15.160:0/456564091 learned_addr learned my addr 172.21.15.160:0/456564091 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-07-08T12:37:44.501 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbcac809700 1 --2- 172.21.15.160:0/456564091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca80b8260 0x7fbca815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.501 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbcad00a700 1 -- 172.21.15.160:0/456564091 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbca80d25c0 msgr2=0x7fbca81f29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.501 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbcad00a700 1 --2- 172.21.15.160:0/456564091 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbca80d25c0 0x7fbca81f29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.501 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbcad00a700 1 -- 172.21.15.160:0/456564091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca80b8260 msgr2=0x7fbca815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.502 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbcad00a700 1 --2- 172.21.15.160:0/456564091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca80b8260 0x7fbca815c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.502 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbcad00a700 1 -- 172.21.15.160:0/456564091 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc940094a0 con 0x7fbca8153030 2023-07-08T12:37:44.502 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbcac809700 1 --2- 172.21.15.160:0/456564091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca80b8260 0x7fbca815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:37:44.502 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbcad00a700 1 --2- 172.21.15.160:0/456564091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca8153030 0x7fbca815edb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbca000eba0 tx=0x7fbca000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:44.502 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbca57fa700 1 -- 172.21.15.160:0/456564091 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca000ca30 con 0x7fbca8153030 2023-07-08T12:37:44.502 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbca57fa700 1 -- 172.21.15.160:0/456564091 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca000f040 con 0x7fbca8153030 2023-07-08T12:37:44.502 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbcaea6d700 1 -- 172.21.15.160:0/456564091 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca815cf80 con 0x7fbca8153030 2023-07-08T12:37:44.502 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbca57fa700 1 -- 172.21.15.160:0/456564091 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca000cdd0 con 0x7fbca8153030 2023-07-08T12:37:44.503 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbcaea6d700 1 -- 172.21.15.160:0/456564091 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca815d540 con 0x7fbca8153030 2023-07-08T12:37:44.503 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbca57fa700 1 -- 172.21.15.160:0/456564091 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbca0014780 con 0x7fbca8153030 2023-07-08T12:37:44.503 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.493+0000 7fbca57fa700 1 -- 172.21.15.160:0/456564091 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fbca001dde0 con 0x7fbca8153030 2023-07-08T12:37:44.503 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.494+0000 7fbcaea6d700 1 -- 172.21.15.160:0/456564091 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca804ea60 con 0x7fbca8153030 2023-07-08T12:37:44.503 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.497+0000 7fbca57fa700 1 -- 172.21.15.160:0/456564091 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fbca003bb00 con 0x7fbca8153030 2023-07-08T12:37:44.622 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.619+0000 7fbcaea6d700 1 -- 172.21.15.160:0/456564091 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd new", "uuid": "92528dc3-7627-4bad-9575-9ceda9430855", "id": 5} v 0) v1 -- 0x7fbca8160140 con 0x7fbca8153030 2023-07-08T12:37:44.631 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.628+0000 7fbca57fa700 1 -- 172.21.15.160:0/456564091 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "92528dc3-7627-4bad-9575-9ceda9430855", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbca002a030 con 0x7fbca8153030 2023-07-08T12:37:44.631 INFO:teuthology.orchestra.run.smithi160.stdout:5 2023-07-08T12:37:44.633 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.629+0000 7fbcaea6d700 1 -- 172.21.15.160:0/456564091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca8153030 msgr2=0x7fbca815edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.633 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.629+0000 7fbcaea6d700 1 --2- 172.21.15.160:0/456564091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca8153030 0x7fbca815edb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbca000eba0 tx=0x7fbca000eeb0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.633 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.629+0000 7fbcaea6d700 1 -- 172.21.15.160:0/456564091 shutdown_connections 2023-07-08T12:37:44.633 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.629+0000 7fbcaea6d700 1 --2- 172.21.15.160:0/456564091 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca8153030 0x7fbca815edb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.634 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.629+0000 7fbcaea6d700 1 --2- 172.21.15.160:0/456564091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbca80b8260 0x7fbca815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.634 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.629+0000 7fbcaea6d700 1 --2- 172.21.15.160:0/456564091 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbca80d25c0 0x7fbca81f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.634 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.629+0000 7fbcaea6d700 1 -- 172.21.15.160:0/456564091 >> 172.21.15.160:0/456564091 conn(0x7fbca80b4d20 msgr2=0x7fbca80b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:44.634 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.629+0000 7fbcaea6d700 1 -- 172.21.15.160:0/456564091 shutdown_connections 2023-07-08T12:37:44.634 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.629+0000 7fbcaea6d700 1 -- 172.21.15.160:0/456564091 wait complete. 2023-07-08T12:37:44.644 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new b4ef7b59-901a-41bd-8705-afeab65c4b14 6 2023-07-08T12:37:44.727 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:44.724+0000 7fc391c1f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-08T12:37:44.755 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:44.753+0000 7f6f35d8c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-08T12:37:44.767 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.764+0000 7f3d0d593700 1 Processor -- start 2023-07-08T12:37:44.767 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.764+0000 7f3d0d593700 1 -- start start 2023-07-08T12:37:44.769 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.765+0000 7f3d0d593700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3d08069f30 0x7f3d080cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.769 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.765+0000 7f3d0d593700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d0815a700 con 0x7f3d0815e780 2023-07-08T12:37:44.769 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.765+0000 7f3d0d593700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d0806f1f0 con 0x7f3d08071550 2023-07-08T12:37:44.769 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.765+0000 7f3d0d593700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0815ed20 con 0x7f3d08069f30 2023-07-08T12:37:44.769 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.765+0000 7f3d06ffd700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3d08069f30 0x7f3d080cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.769 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.765+0000 7f3d06ffd700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3d08069f30 0x7f3d080cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:38360/0 (socket says 172.21.15.160:38360) 2023-07-08T12:37:44.770 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.765+0000 7f3d077fe700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f3d08071550 0x7f3d0806f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:42936/0 (socket says 172.21.15.160:42936) 2023-07-08T12:37:44.770 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.765+0000 7f3d06ffd700 1 -- 172.21.15.160:0/3661810066 learned_addr learned my addr 172.21.15.160:0/3661810066 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-07-08T12:37:44.770 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.766+0000 7f3d06ffd700 1 -- 172.21.15.160:0/3661810066 >> v1:172.21.15.160:6789/0 conn(0x7f3d08071550 legacy=0x7f3d0806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.770 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.766+0000 7f3d06ffd700 1 -- 172.21.15.160:0/3661810066 >> v1:172.21.15.12:6789/0 conn(0x7f3d0815e780 legacy=0x7f3d0806a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.770 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.766+0000 7f3d06ffd700 1 -- 172.21.15.160:0/3661810066 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d080ca6c0 con 0x7f3d08069f30 2023-07-08T12:37:44.770 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.766+0000 7f3d06ffd700 1 --2- 172.21.15.160:0/3661810066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d08069f30 0x7f3d080cbaf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0098f0 tx=0x7f3cfc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=836c575aee06e202 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:44.770 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.766+0000 7f3d05ffb700 1 -- 172.21.15.160:0/3661810066 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00b8a0 con 0x7f3d08069f30 2023-07-08T12:37:44.771 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.766+0000 7f3d05ffb700 1 -- 172.21.15.160:0/3661810066 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc00ba00 con 0x7f3d08069f30 2023-07-08T12:37:44.771 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.766+0000 7f3d05ffb700 1 -- 172.21.15.160:0/3661810066 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00bd90 con 0x7f3d08069f30 2023-07-08T12:37:44.773 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.767+0000 7f3d0d593700 1 -- 172.21.15.160:0/3661810066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d08069f30 msgr2=0x7f3d080cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.774 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.767+0000 7f3d0d593700 1 --2- 172.21.15.160:0/3661810066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d08069f30 0x7f3d080cbaf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0098f0 tx=0x7f3cfc009c00 comp rx=0 tx=0).stop 2023-07-08T12:37:44.774 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.767+0000 7f3d0d593700 1 -- 172.21.15.160:0/3661810066 shutdown_connections 2023-07-08T12:37:44.774 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.767+0000 7f3d0d593700 1 --2- 172.21.15.160:0/3661810066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d08069f30 0x7f3d080cbaf0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.774 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.767+0000 7f3d0d593700 1 -- 172.21.15.160:0/3661810066 >> 172.21.15.160:0/3661810066 conn(0x7f3d080cc390 msgr2=0x7f3d080cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:44.774 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.767+0000 7f3d0d593700 1 -- 172.21.15.160:0/3661810066 shutdown_connections 2023-07-08T12:37:44.774 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.767+0000 7f3d0d593700 1 -- 172.21.15.160:0/3661810066 wait complete. 2023-07-08T12:37:44.774 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.768+0000 7f3d0d593700 1 Processor -- start 2023-07-08T12:37:44.775 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.768+0000 7f3d0d593700 1 -- start start 2023-07-08T12:37:44.775 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d0d593700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d08069f30 0x7f3d0814de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.775 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d0d593700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d08071550 0x7f3d080c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.775 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d0d593700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3d0815e780 0x7f3d080c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:44.775 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d0d593700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d080c8cf0 con 0x7f3d08069f30 2023-07-08T12:37:44.775 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d0d593700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0814e360 con 0x7f3d08071550 2023-07-08T12:37:44.775 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d0d593700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d080c8070 con 0x7f3d0815e780 2023-07-08T12:37:44.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d077fe700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3d0815e780 0x7f3d080c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d077fe700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3d0815e780 0x7f3d080c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:46264/0 (socket says 172.21.15.160:46264) 2023-07-08T12:37:44.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d06ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d08069f30 0x7f3d0814de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d077fe700 1 -- 172.21.15.160:0/3585614837 learned_addr learned my addr 172.21.15.160:0/3585614837 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-07-08T12:37:44.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d067fc700 1 --2- 172.21.15.160:0/3585614837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d08071550 0x7f3d080c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:44.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d077fe700 1 -- 172.21.15.160:0/3585614837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d08071550 msgr2=0x7f3d080c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d077fe700 1 --2- 172.21.15.160:0/3585614837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d08071550 0x7f3d080c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d077fe700 1 -- 172.21.15.160:0/3585614837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d08069f30 msgr2=0x7f3d0814de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.777 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d077fe700 1 --2- 172.21.15.160:0/3585614837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d08069f30 0x7f3d0814de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.777 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d077fe700 1 -- 172.21.15.160:0/3585614837 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc009550 con 0x7f3d0815e780 2023-07-08T12:37:44.777 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d06ffd700 1 --2- 172.21.15.160:0/3585614837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d08069f30 0x7f3d0814de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:37:44.777 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.769+0000 7f3d067fc700 1 --2- 172.21.15.160:0/3585614837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d08071550 0x7f3d080c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:37:44.777 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.770+0000 7f3d077fe700 1 --2- 172.21.15.160:0/3585614837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3d0815e780 0x7f3d080c9110 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000cc00 tx=0x7f3d0000cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.770+0000 7f3cef7fe700 1 -- 172.21.15.160:0/3585614837 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0000aa40 con 0x7f3d0815e780 2023-07-08T12:37:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.770+0000 7f3cef7fe700 1 -- 172.21.15.160:0/3585614837 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0000aba0 con 0x7f3d0815e780 2023-07-08T12:37:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.770+0000 7f3cef7fe700 1 -- 172.21.15.160:0/3585614837 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00007070 con 0x7f3d0815e780 2023-07-08T12:37:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.770+0000 7f3d0d593700 1 -- 172.21.15.160:0/3585614837 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d080c8350 con 0x7f3d0815e780 2023-07-08T12:37:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.770+0000 7f3d0d593700 1 -- 172.21.15.160:0/3585614837 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d080c8910 con 0x7f3d0815e780 2023-07-08T12:37:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.770+0000 7f3cef7fe700 1 -- 172.21.15.160:0/3585614837 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3d000074d0 con 0x7f3d0815e780 2023-07-08T12:37:44.778 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.770+0000 7f3cef7fe700 1 -- 172.21.15.160:0/3585614837 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f3d00012400 con 0x7f3d0815e780 2023-07-08T12:37:44.779 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.771+0000 7f3d0d593700 1 -- 172.21.15.160:0/3585614837 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d080c8e30 con 0x7f3d0815e780 2023-07-08T12:37:44.779 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.774+0000 7f3cef7fe700 1 -- 172.21.15.160:0/3585614837 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f3d080c8e30 con 0x7f3d0815e780 2023-07-08T12:37:44.860 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.858+0000 7f3d0d593700 1 -- 172.21.15.160:0/3585614837 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b4ef7b59-901a-41bd-8705-afeab65c4b14", "id": 6} v 0) v1 -- 0x7f3d080b53c0 con 0x7f3d0815e780 2023-07-08T12:37:44.868 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.866+0000 7f3cef7fe700 1 -- 172.21.15.160:0/3585614837 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b4ef7b59-901a-41bd-8705-afeab65c4b14", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3d00021060 con 0x7f3d0815e780 2023-07-08T12:37:44.869 INFO:teuthology.orchestra.run.smithi160.stdout:6 2023-07-08T12:37:44.870 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.867+0000 7f3d0d593700 1 -- 172.21.15.160:0/3585614837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3d0815e780 msgr2=0x7f3d080c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:44.870 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.867+0000 7f3d0d593700 1 --2- 172.21.15.160:0/3585614837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3d0815e780 0x7f3d080c9110 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000cc00 tx=0x7f3d0000cf10 comp rx=0 tx=0).stop 2023-07-08T12:37:44.870 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.867+0000 7f3d0d593700 1 -- 172.21.15.160:0/3585614837 shutdown_connections 2023-07-08T12:37:44.870 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.867+0000 7f3d0d593700 1 --2- 172.21.15.160:0/3585614837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3d0815e780 0x7f3d080c9110 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.870 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.867+0000 7f3d0d593700 1 --2- 172.21.15.160:0/3585614837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d08069f30 0x7f3d0814de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.870 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.867+0000 7f3d0d593700 1 --2- 172.21.15.160:0/3585614837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d08071550 0x7f3d080c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:44.871 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.867+0000 7f3d0d593700 1 -- 172.21.15.160:0/3585614837 >> 172.21.15.160:0/3585614837 conn(0x7f3d080cc390 msgr2=0x7f3d080729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:44.871 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.867+0000 7f3d0d593700 1 -- 172.21.15.160:0/3585614837 shutdown_connections 2023-07-08T12:37:44.871 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:44.867+0000 7f3d0d593700 1 -- 172.21.15.160:0/3585614837 wait complete. 2023-07-08T12:37:44.881 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new 06bf1e54-02a7-48c8-a72e-9364b2c575d4 7 2023-07-08T12:37:44.894 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:44.891+0000 7fc391c1f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-08T12:37:44.916 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:44.914+0000 7f6f35d8c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-08T12:37:45.003 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.000+0000 7fb8cbbfa700 1 Processor -- start 2023-07-08T12:37:45.003 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.001+0000 7fb8cbbfa700 1 -- start start 2023-07-08T12:37:45.005 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8cbbfa700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb8c40d0170 0x7fb8c41607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:45.005 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8cbbfa700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40cd360 0x7fb8c40cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:45.005 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8cbbfa700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c40cdd20 con 0x7fb8c415a0d0 2023-07-08T12:37:45.005 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8cbbfa700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c41603a0 con 0x7fb8c40d0170 2023-07-08T12:37:45.006 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8cbbfa700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40d05d0 con 0x7fb8c40cd360 2023-07-08T12:37:45.006 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8ca197700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40cd360 0x7fb8c40cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:45.006 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8ca197700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40cd360 0x7fb8c40cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:46270/0 (socket says 172.21.15.160:46270) 2023-07-08T12:37:45.006 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8ca197700 1 -- 172.21.15.160:0/3377328606 learned_addr learned my addr 172.21.15.160:0/3377328606 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-07-08T12:37:45.006 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8c9195700 1 --2- 172.21.15.160:0/3377328606 >> v2:172.21.15.12:3300/0 conn(0x7fb8c40d0170 0x7fb8c41607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:45.006 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8ca197700 1 -- 172.21.15.160:0/3377328606 >> v2:172.21.15.12:3300/0 conn(0x7fb8c40d0170 msgr2=0x7fb8c41607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:37:45.006 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8ca197700 1 --2- 172.21.15.160:0/3377328606 >> v2:172.21.15.12:3300/0 conn(0x7fb8c40d0170 0x7fb8c41607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.006 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8ca197700 1 -- 172.21.15.160:0/3377328606 >> v1:172.21.15.12:6789/0 conn(0x7fb8c415a0d0 legacy=0x7fb8c415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:45.007 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8ca197700 1 -- 172.21.15.160:0/3377328606 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c4160000 con 0x7fb8c40cd360 2023-07-08T12:37:45.007 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8ca197700 1 --2- 172.21.15.160:0/3377328606 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40cd360 0x7fb8c40cd7e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009e50 tx=0x7fb8bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbd894e920d155df server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:45.007 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8c8994700 1 -- 172.21.15.160:0/3377328606 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00e070 con 0x7fb8c40cd360 2023-07-08T12:37:45.007 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8c8994700 1 -- 172.21.15.160:0/3377328606 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8bc00b910 con 0x7fb8c40cd360 2023-07-08T12:37:45.007 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.002+0000 7fb8c8994700 1 -- 172.21.15.160:0/3377328606 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00bcc0 con 0x7fb8c40cd360 2023-07-08T12:37:45.010 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.003+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/3377328606 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40cd360 msgr2=0x7fb8c40cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:45.010 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.003+0000 7fb8cbbfa700 1 --2- 172.21.15.160:0/3377328606 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40cd360 0x7fb8c40cd7e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009e50 tx=0x7fb8bc00b330 comp rx=0 tx=0).stop 2023-07-08T12:37:45.010 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.003+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/3377328606 shutdown_connections 2023-07-08T12:37:45.010 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.003+0000 7fb8cbbfa700 1 --2- 172.21.15.160:0/3377328606 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40cd360 0x7fb8c40cd7e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.010 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.003+0000 7fb8cbbfa700 1 --2- 172.21.15.160:0/3377328606 >> v2:172.21.15.12:3300/0 conn(0x7fb8c40d0170 0x7fb8c41607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.003+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/3377328606 >> 172.21.15.160:0/3377328606 conn(0x7fb8c406e6c0 msgr2=0x7fb8c406ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.003+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/3377328606 shutdown_connections 2023-07-08T12:37:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.003+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/3377328606 wait complete. 2023-07-08T12:37:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.004+0000 7fb8cbbfa700 1 Processor -- start 2023-07-08T12:37:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.004+0000 7fb8cbbfa700 1 -- start start 2023-07-08T12:37:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8cbbfa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c40cd360 0x7fb8c41f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:45.011 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8cbbfa700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40d0170 0x7fb8c414edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:45.012 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8cbbfa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8c415a0d0 0x7fb8c406c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:45.012 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8cbbfa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40b5360 con 0x7fb8c40cd360 2023-07-08T12:37:45.013 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8cbbfa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c414e0e0 con 0x7fb8c415a0d0 2023-07-08T12:37:45.013 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8cbbfa700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c414e940 con 0x7fb8c40d0170 2023-07-08T12:37:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8c9195700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40d0170 0x7fb8c414edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8c9195700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40d0170 0x7fb8c414edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:46274/0 (socket says 172.21.15.160:46274) 2023-07-08T12:37:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8c9195700 1 -- 172.21.15.160:0/2493341466 learned_addr learned my addr 172.21.15.160:0/2493341466 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-07-08T12:37:45.014 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8c9996700 1 --2- 172.21.15.160:0/2493341466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c40cd360 0x7fb8c41f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:45.015 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8ca197700 1 --2- 172.21.15.160:0/2493341466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8c415a0d0 0x7fb8c406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:45.015 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8c9195700 1 -- 172.21.15.160:0/2493341466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8c415a0d0 msgr2=0x7fb8c406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:45.015 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8c9195700 1 --2- 172.21.15.160:0/2493341466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8c415a0d0 0x7fb8c406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.016 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8c9195700 1 -- 172.21.15.160:0/2493341466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c40cd360 msgr2=0x7fb8c41f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:45.016 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8c9195700 1 --2- 172.21.15.160:0/2493341466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c40cd360 0x7fb8c41f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.016 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8c9195700 1 -- 172.21.15.160:0/2493341466 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8bc009b30 con 0x7fb8c40d0170 2023-07-08T12:37:45.016 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.005+0000 7fb8c9996700 1 --2- 172.21.15.160:0/2493341466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c40cd360 0x7fb8c41f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:37:45.017 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.006+0000 7fb8c9195700 1 --2- 172.21.15.160:0/2493341466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40d0170 0x7fb8c414edd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400e9b0 tx=0x7fb8b400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:45.017 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.006+0000 7fb8ba7fc700 1 -- 172.21.15.160:0/2493341466 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4009710 con 0x7fb8c40d0170 2023-07-08T12:37:45.017 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.006+0000 7fb8ba7fc700 1 -- 172.21.15.160:0/2493341466 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b400c850 con 0x7fb8c40d0170 2023-07-08T12:37:45.018 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.006+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/2493341466 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c40b5080 con 0x7fb8c40d0170 2023-07-08T12:37:45.018 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.006+0000 7fb8ba7fc700 1 -- 172.21.15.160:0/2493341466 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400cbc0 con 0x7fb8c40d0170 2023-07-08T12:37:45.018 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.006+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/2493341466 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c414e5d0 con 0x7fb8c40d0170 2023-07-08T12:37:45.018 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.006+0000 7fb8ba7fc700 1 -- 172.21.15.160:0/2493341466 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb8b4005d20 con 0x7fb8c40d0170 2023-07-08T12:37:45.019 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.006+0000 7fb8ba7fc700 1 -- 172.21.15.160:0/2493341466 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fb8b40067a0 con 0x7fb8c40d0170 2023-07-08T12:37:45.019 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.007+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/2493341466 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c404ea60 con 0x7fb8c40d0170 2023-07-08T12:37:45.020 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.009+0000 7fb8ba7fc700 1 -- 172.21.15.160:0/2493341466 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fb8b4021060 con 0x7fb8c40d0170 2023-07-08T12:37:45.094 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.091+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/2493341466 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd new", "uuid": "06bf1e54-02a7-48c8-a72e-9364b2c575d4", "id": 7} v 0) v1 -- 0x7fb8c414ea80 con 0x7fb8c40d0170 2023-07-08T12:37:45.102 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.100+0000 7fb8ba7fc700 1 -- 172.21.15.160:0/2493341466 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "06bf1e54-02a7-48c8-a72e-9364b2c575d4", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb8b4017070 con 0x7fb8c40d0170 2023-07-08T12:37:45.103 INFO:teuthology.orchestra.run.smithi160.stdout:7 2023-07-08T12:37:45.105 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.101+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/2493341466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40d0170 msgr2=0x7fb8c414edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:45.105 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.101+0000 7fb8cbbfa700 1 --2- 172.21.15.160:0/2493341466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40d0170 0x7fb8c414edd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400e9b0 tx=0x7fb8b400ecc0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.106 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.101+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/2493341466 shutdown_connections 2023-07-08T12:37:45.106 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.101+0000 7fb8cbbfa700 1 --2- 172.21.15.160:0/2493341466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8c415a0d0 0x7fb8c406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.106 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.101+0000 7fb8cbbfa700 1 --2- 172.21.15.160:0/2493341466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c40cd360 0x7fb8c41f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.106 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.101+0000 7fb8cbbfa700 1 --2- 172.21.15.160:0/2493341466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8c40d0170 0x7fb8c414edd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.107 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.101+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/2493341466 >> 172.21.15.160:0/2493341466 conn(0x7fb8c406e6c0 msgr2=0x7fb8c41585b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:45.107 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.101+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/2493341466 shutdown_connections 2023-07-08T12:37:45.108 INFO:teuthology.orchestra.run.smithi160.stderr:2023-07-08T12:37:45.101+0000 7fb8cbbfa700 1 -- 172.21.15.160:0/2493341466 wait complete. 2023-07-08T12:37:45.116 INFO:tasks.ceph.osd.0:Restarting daemon 2023-07-08T12:37:45.117 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-07-08T12:37:45.120 INFO:tasks.ceph.osd.0:Started 2023-07-08T12:37:45.121 INFO:tasks.ceph.osd.1:Restarting daemon 2023-07-08T12:37:45.121 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-07-08T12:37:45.124 INFO:tasks.ceph.osd.1:Started 2023-07-08T12:37:45.125 INFO:tasks.ceph.osd.2:Restarting daemon 2023-07-08T12:37:45.125 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-07-08T12:37:45.127 INFO:tasks.ceph.osd.2:Started 2023-07-08T12:37:45.128 INFO:tasks.ceph.osd.3:Restarting daemon 2023-07-08T12:37:45.128 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-07-08T12:37:45.130 INFO:tasks.ceph.osd.3:Started 2023-07-08T12:37:45.130 INFO:tasks.ceph.osd.4:Restarting daemon 2023-07-08T12:37:45.131 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-07-08T12:37:45.158 INFO:tasks.ceph.osd.4:Started 2023-07-08T12:37:45.159 INFO:tasks.ceph.osd.5:Restarting daemon 2023-07-08T12:37:45.159 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-07-08T12:37:45.161 INFO:tasks.ceph.osd.5:Started 2023-07-08T12:37:45.162 INFO:tasks.ceph.osd.6:Restarting daemon 2023-07-08T12:37:45.162 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-07-08T12:37:45.164 INFO:tasks.ceph.osd.6:Started 2023-07-08T12:37:45.164 INFO:tasks.ceph.osd.7:Restarting daemon 2023-07-08T12:37:45.164 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-07-08T12:37:45.166 INFO:tasks.ceph.osd.7:Started 2023-07-08T12:37:45.167 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:37:45.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.313+0000 7f29532ec700 1 Processor -- start 2023-07-08T12:37:45.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.314+0000 7f29532ec700 1 -- start start 2023-07-08T12:37:45.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.314+0000 7f29532ec700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f294c160790 0x7f294c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:45.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.314+0000 7f29532ec700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c15c890 con 0x7f294c069f30 2023-07-08T12:37:45.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.314+0000 7f29532ec700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c06f620 con 0x7f294c159c30 2023-07-08T12:37:45.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.314+0000 7f29532ec700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c15c730 con 0x7f294c160790 2023-07-08T12:37:45.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.315+0000 7f2950887700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f294c160790 0x7f294c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:45.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.315+0000 7f2951889700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f294c069f30 0x7f294c15cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35918/0 (socket says 172.21.15.12:35918) 2023-07-08T12:37:45.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.315+0000 7f2951889700 1 -- 172.21.15.12:0/2382101117 learned_addr learned my addr 172.21.15.12:0/2382101117 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:37:45.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.315+0000 7f2950887700 1 -- 172.21.15.12:0/2382101117 >> v1:172.21.15.160:6789/0 conn(0x7f294c159c30 legacy=0x7f294c06ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:45.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.315+0000 7f2950887700 1 -- 172.21.15.12:0/2382101117 >> v1:172.21.15.12:6789/0 conn(0x7f294c069f30 legacy=0x7f294c15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:45.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.315+0000 7f2950887700 1 -- 172.21.15.12:0/2382101117 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c160c90 con 0x7f294c160790 2023-07-08T12:37:45.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.315+0000 7f2950887700 1 --2- 172.21.15.12:0/2382101117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f294c160790 0x7f294c0cf050 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f293c009840 tx=0x7f293c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23227aafccbfec91 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:45.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.315+0000 7f2943fff700 1 -- 172.21.15.12:0/2382101117 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c011830 con 0x7f294c160790 2023-07-08T12:37:45.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.315+0000 7f2943fff700 1 -- 172.21.15.12:0/2382101117 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c011990 con 0x7f294c160790 2023-07-08T12:37:45.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.315+0000 7f2943fff700 1 -- 172.21.15.12:0/2382101117 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c011d20 con 0x7f294c160790 2023-07-08T12:37:45.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.316+0000 7f29532ec700 1 -- 172.21.15.12:0/2382101117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f294c160790 msgr2=0x7f294c0cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:45.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.316+0000 7f29532ec700 1 --2- 172.21.15.12:0/2382101117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f294c160790 0x7f294c0cf050 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f293c009840 tx=0x7f293c009b50 comp rx=0 tx=0).stop 2023-07-08T12:37:45.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.316+0000 7f29532ec700 1 -- 172.21.15.12:0/2382101117 shutdown_connections 2023-07-08T12:37:45.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.316+0000 7f29532ec700 1 --2- 172.21.15.12:0/2382101117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f294c160790 0x7f294c0cf050 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.316+0000 7f29532ec700 1 -- 172.21.15.12:0/2382101117 >> 172.21.15.12:0/2382101117 conn(0x7f294c0cc290 msgr2=0x7f294c0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:45.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.316+0000 7f29532ec700 1 -- 172.21.15.12:0/2382101117 shutdown_connections 2023-07-08T12:37:45.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.316+0000 7f29532ec700 1 -- 172.21.15.12:0/2382101117 wait complete. 2023-07-08T12:37:45.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.317+0000 7f29532ec700 1 Processor -- start 2023-07-08T12:37:45.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.317+0000 7f29532ec700 1 -- start start 2023-07-08T12:37:45.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f29532ec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f294c069f30 0x7f294c14e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:45.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f29532ec700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f294c159c30 0x7f294c0b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:45.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f29532ec700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f294c160790 0x7f294c0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:45.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f29532ec700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0c3340 con 0x7f294c159c30 2023-07-08T12:37:45.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f29532ec700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0c5a90 con 0x7f294c069f30 2023-07-08T12:37:45.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f29532ec700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0c4e80 con 0x7f294c160790 2023-07-08T12:37:45.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f2950887700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f294c159c30 0x7f294c0b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:45.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f2950887700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f294c159c30 0x7f294c0b17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48526/0 (socket says 172.21.15.12:48526) 2023-07-08T12:37:45.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f2950887700 1 -- 172.21.15.12:0/930846196 learned_addr learned my addr 172.21.15.12:0/930846196 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:37:45.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f2951088700 1 --2- 172.21.15.12:0/930846196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f294c069f30 0x7f294c14e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:45.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f2951889700 1 --2- 172.21.15.12:0/930846196 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f294c160790 0x7f294c0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:45.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f2950887700 1 -- 172.21.15.12:0/930846196 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f294c160790 msgr2=0x7f294c0b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:45.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f2950887700 1 --2- 172.21.15.12:0/930846196 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f294c160790 0x7f294c0b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f2950887700 1 -- 172.21.15.12:0/930846196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f294c069f30 msgr2=0x7f294c14e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:45.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f2950887700 1 --2- 172.21.15.12:0/930846196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f294c069f30 0x7f294c14e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f2950887700 1 -- 172.21.15.12:0/930846196 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c0094a0 con 0x7f294c159c30 2023-07-08T12:37:45.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.318+0000 7f2950887700 1 --2- 172.21.15.12:0/930846196 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f294c159c30 0x7f294c0b17d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f293c000c00 tx=0x7f293c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:45.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.319+0000 7f2941ffb700 1 -- 172.21.15.12:0/930846196 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c0038a0 con 0x7f294c159c30 2023-07-08T12:37:45.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.319+0000 7f2941ffb700 1 -- 172.21.15.12:0/930846196 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c01e070 con 0x7f294c159c30 2023-07-08T12:37:45.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.319+0000 7f2941ffb700 1 -- 172.21.15.12:0/930846196 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c003c60 con 0x7f294c159c30 2023-07-08T12:37:45.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.319+0000 7f29532ec700 1 -- 172.21.15.12:0/930846196 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c0c5160 con 0x7f294c159c30 2023-07-08T12:37:45.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.319+0000 7f29532ec700 1 -- 172.21.15.12:0/930846196 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c0c5720 con 0x7f294c159c30 2023-07-08T12:37:45.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.319+0000 7f2941ffb700 1 -- 172.21.15.12:0/930846196 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f293c02d750 con 0x7f294c159c30 2023-07-08T12:37:45.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.319+0000 7f2941ffb700 1 -- 172.21.15.12:0/930846196 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f293c01aaf0 con 0x7f294c159c30 2023-07-08T12:37:45.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.320+0000 7f29532ec700 1 -- 172.21.15.12:0/930846196 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c0cd740 con 0x7f294c159c30 2023-07-08T12:37:45.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.322+0000 7f2941ffb700 1 -- 172.21.15.12:0/930846196 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f294c0cd740 con 0x7f294c159c30 2023-07-08T12:37:45.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.393+0000 7f29532ec700 1 -- 172.21.15.12:0/930846196 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f294c04ea60 con 0x7f294c159c30 2023-07-08T12:37:45.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.394+0000 7f2941ffb700 1 -- 172.21.15.12:0/930846196 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f293c020070 con 0x7f294c159c30 2023-07-08T12:37:45.397 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:37:45.397 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":10,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:37:45.094491+0000","last_up_change":"0.000000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:37:45.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.395+0000 7f29532ec700 1 -- 172.21.15.12:0/930846196 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f294c159c30 msgr2=0x7f294c0b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.395+0000 7f29532ec700 1 --2- 172.21.15.12:0/930846196 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f294c159c30 0x7f294c0b17d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f293c000c00 tx=0x7f293c004750 comp rx=0 tx=0).stop 2023-07-08T12:37:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.395+0000 7f29532ec700 1 -- 172.21.15.12:0/930846196 shutdown_connections 2023-07-08T12:37:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.395+0000 7f29532ec700 1 --2- 172.21.15.12:0/930846196 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f294c160790 0x7f294c0b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.395+0000 7f29532ec700 1 --2- 172.21.15.12:0/930846196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f294c069f30 0x7f294c14e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.395+0000 7f29532ec700 1 --2- 172.21.15.12:0/930846196 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f294c159c30 0x7f294c0b17d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:45.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.395+0000 7f29532ec700 1 -- 172.21.15.12:0/930846196 >> 172.21.15.12:0/930846196 conn(0x7f294c0cc290 msgr2=0x7f294c06a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:45.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.395+0000 7f29532ec700 1 -- 172.21.15.12:0/930846196 shutdown_connections 2023-07-08T12:37:45.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:37:45.395+0000 7f29532ec700 1 -- 172.21.15.12:0/930846196 wait complete. 2023-07-08T12:37:45.410 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-07-08T12:37:45.410 INFO:tasks.ceph:Waiting for OSDs to come up 2023-07-08T12:37:45.625 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:45.622+0000 7fc391c1f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-08T12:37:45.685 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:45.683+0000 7f6f35d8c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-08T12:37:45.702 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:45.699+0000 7fc391c1f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-08T12:37:45.739 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-07-08T12:37:45.737+0000 7f8404f78640 -1 Falling back to public interface 2023-07-08T12:37:45.745 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-07-08T12:37:45.742+0000 7fe6cc782640 -1 Falling back to public interface 2023-07-08T12:37:45.747 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-07-08T12:37:45.745+0000 7fcf3b222640 -1 Falling back to public interface 2023-07-08T12:37:45.767 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:45.765+0000 7f6f35d8c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-08T12:37:45.768 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-07-08T12:37:45.765+0000 7f58a5c98640 -1 Falling back to public interface 2023-07-08T12:37:45.775 INFO:tasks.ceph.osd.6.smithi160.stderr:2023-07-08T12:37:45.772+0000 7fb3cb5ab640 -1 Falling back to public interface 2023-07-08T12:37:45.787 INFO:tasks.ceph.osd.4.smithi160.stderr:2023-07-08T12:37:45.784+0000 7efd7a32f640 -1 Falling back to public interface 2023-07-08T12:37:45.787 INFO:tasks.ceph.osd.7.smithi160.stderr:2023-07-08T12:37:45.784+0000 7f0f5ad95640 -1 Falling back to public interface 2023-07-08T12:37:45.788 INFO:tasks.ceph.osd.7.smithi160.stderr:2023-07-08T12:37:45.786+0000 7f0f52153700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6818/0: (98) Address already in use 2023-07-08T12:37:45.789 INFO:tasks.ceph.osd.5.smithi160.stderr:2023-07-08T12:37:45.786+0000 7f3d40c65640 -1 Falling back to public interface 2023-07-08T12:37:45.847 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:45.843+0000 7fc391c1f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-08T12:37:45.914 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:45.911+0000 7f6f35d8c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-08T12:37:46.079 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:46.076+0000 7fc391c1f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-08T12:37:46.153 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:46.151+0000 7fc391c1f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-08T12:37:46.158 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:46.156+0000 7f6f35d8c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-08T12:37:46.215 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-08T12:37:46.236 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:46.233+0000 7f6f35d8c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-08T12:37:46.333 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.331+0000 7f5d2259f700 1 Processor -- start 2023-07-08T12:37:46.333 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.331+0000 7f5d2259f700 1 -- start start 2023-07-08T12:37:46.335 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d2259f700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5d1c06cbd0 0x7f5d1c06cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:46.335 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d2259f700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1c15ff80 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.335 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d2259f700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1c071c00 con 0x7f5d1c0cd190 2023-07-08T12:37:46.335 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d2259f700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d1c06d4e0 con 0x7f5d1c06cbd0 2023-07-08T12:37:46.335 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d1b7fe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5d1c0ccd40 0x7f5d1c070de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35926/0 (socket says 172.21.15.12:35926) 2023-07-08T12:37:46.336 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d1b7fe700 1 -- 172.21.15.12:0/2303397616 learned_addr learned my addr 172.21.15.12:0/2303397616 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:37:46.336 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d1bfff700 1 --2- 172.21.15.12:0/2303397616 >> v2:172.21.15.160:3300/0 conn(0x7f5d1c06cbd0 0x7f5d1c06cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:46.336 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d1affd700 1 -- 172.21.15.12:0/2303397616 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718268953 0 0) 0x7f5d1c15ff80 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.336 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d1affd700 1 -- 172.21.15.12:0/2303397616 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d040035b0 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.337 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d1affd700 1 -- 172.21.15.12:0/2303397616 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346994761 0 0) 0x7f5d1c071c00 con 0x7f5d1c0cd190 2023-07-08T12:37:46.337 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d1affd700 1 -- 172.21.15.12:0/2303397616 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d1c15ff80 con 0x7f5d1c0cd190 2023-07-08T12:37:46.337 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d1bfff700 1 -- 172.21.15.12:0/2303397616 >> v1:172.21.15.160:6789/0 conn(0x7f5d1c0cd190 legacy=0x7f5d1c0714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:46.337 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d1bfff700 1 -- 172.21.15.12:0/2303397616 >> v1:172.21.15.12:6789/0 conn(0x7f5d1c0ccd40 legacy=0x7f5d1c070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:46.337 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.332+0000 7f5d1bfff700 1 -- 172.21.15.12:0/2303397616 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c1601d0 con 0x7f5d1c06cbd0 2023-07-08T12:37:46.337 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.333+0000 7f5d1bfff700 1 --2- 172.21.15.12:0/2303397616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d1c06cbd0 0x7f5d1c06cfa0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009840 tx=0x7f5d0c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa4ced1e9798b46 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:46.337 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.333+0000 7f5d1affd700 1 -- 172.21.15.12:0/2303397616 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c011830 con 0x7f5d1c06cbd0 2023-07-08T12:37:46.337 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.333+0000 7f5d1affd700 1 -- 172.21.15.12:0/2303397616 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0c011990 con 0x7f5d1c06cbd0 2023-07-08T12:37:46.338 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.333+0000 7f5d1affd700 1 -- 172.21.15.12:0/2303397616 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c011d20 con 0x7f5d1c06cbd0 2023-07-08T12:37:46.340 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.334+0000 7f5d2259f700 1 -- 172.21.15.12:0/2303397616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d1c06cbd0 msgr2=0x7f5d1c06cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:46.340 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.334+0000 7f5d2259f700 1 --2- 172.21.15.12:0/2303397616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d1c06cbd0 0x7f5d1c06cfa0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009840 tx=0x7f5d0c009b50 comp rx=0 tx=0).stop 2023-07-08T12:37:46.340 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.334+0000 7f5d2259f700 1 -- 172.21.15.12:0/2303397616 shutdown_connections 2023-07-08T12:37:46.340 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.334+0000 7f5d2259f700 1 --2- 172.21.15.12:0/2303397616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d1c06cbd0 0x7f5d1c06cfa0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:46.340 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.334+0000 7f5d2259f700 1 -- 172.21.15.12:0/2303397616 >> 172.21.15.12:0/2303397616 conn(0x7f5d1c0b1920 msgr2=0x7f5d1c0cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:46.340 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.334+0000 7f5d2259f700 1 -- 172.21.15.12:0/2303397616 shutdown_connections 2023-07-08T12:37:46.341 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.334+0000 7f5d2259f700 1 -- 172.21.15.12:0/2303397616 wait complete. 2023-07-08T12:37:46.341 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.335+0000 7f5d2259f700 1 Processor -- start 2023-07-08T12:37:46.341 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.336+0000 7f5d2259f700 1 -- start start 2023-07-08T12:37:46.341 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.336+0000 7f5d2259f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d1c06cbd0 0x7f5d1c0c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:46.341 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.336+0000 7f5d2259f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d1c0ccd40 0x7f5d1c144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:46.341 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.336+0000 7f5d2259f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d1c0cd190 0x7f5d1c0c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:46.341 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.336+0000 7f5d2259f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0c7b00 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.342 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.336+0000 7f5d2259f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0c71d0 con 0x7f5d1c06cbd0 2023-07-08T12:37:46.342 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d2259f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0c74b0 con 0x7f5d1c0cd190 2023-07-08T12:37:46.342 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d1b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d1c0ccd40 0x7f5d1c144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:46.342 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d1b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d1c0ccd40 0x7f5d1c144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48538/0 (socket says 172.21.15.12:48538) 2023-07-08T12:37:46.343 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d20b3c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d1c0cd190 0x7f5d1c0c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:46.343 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d1b7fe700 1 -- 172.21.15.12:0/1589122987 learned_addr learned my addr 172.21.15.12:0/1589122987 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:37:46.343 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d1bfff700 1 --2- 172.21.15.12:0/1589122987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d1c06cbd0 0x7f5d1c0c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:46.343 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d1b7fe700 1 -- 172.21.15.12:0/1589122987 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d1c0cd190 msgr2=0x7f5d1c0c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:46.343 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d1b7fe700 1 --2- 172.21.15.12:0/1589122987 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d1c0cd190 0x7f5d1c0c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:46.343 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d1b7fe700 1 -- 172.21.15.12:0/1589122987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d1c06cbd0 msgr2=0x7f5d1c0c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:46.343 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d1b7fe700 1 --2- 172.21.15.12:0/1589122987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d1c06cbd0 0x7f5d1c0c2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:46.343 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d1b7fe700 1 -- 172.21.15.12:0/1589122987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c0094a0 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.344 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d20b3c700 1 --2- 172.21.15.12:0/1589122987 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d1c0cd190 0x7f5d1c0c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:37:46.344 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d1b7fe700 1 --2- 172.21.15.12:0/1589122987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d1c0ccd40 0x7f5d1c144110 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000c700 tx=0x7f5d1000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:46.344 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d18ff9700 1 -- 172.21.15.12:0/1589122987 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10005d70 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.344 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d18ff9700 1 -- 172.21.15.12:0/1589122987 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d100070d0 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.344 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d2259f700 1 -- 172.21.15.12:0/1589122987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1c0c7790 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.344 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d18ff9700 1 -- 172.21.15.12:0/1589122987 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10007440 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.344 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.337+0000 7f5d2259f700 1 -- 172.21.15.12:0/1589122987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1c0c7f90 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.345 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.338+0000 7f5d18ff9700 1 -- 172.21.15.12:0/1589122987 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5d1001c440 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.345 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.338+0000 7f5d18ff9700 1 -- 172.21.15.12:0/1589122987 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f5d10012ad0 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.345 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.338+0000 7f5d2259f700 1 -- 172.21.15.12:0/1589122987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1c0c8210 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.345 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.341+0000 7f5d18ff9700 1 -- 172.21.15.12:0/1589122987 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f5d1c0c8210 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.414 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.412+0000 7f5d2259f700 1 -- 172.21.15.12:0/1589122987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d1c073970 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.414 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.412+0000 7f5d18ff9700 1 -- 172.21.15.12:0/1589122987 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f5d10021060 con 0x7f5d1c0ccd40 2023-07-08T12:37:46.415 INFO:teuthology.misc.health.smithi012.stdout: 2023-07-08T12:37:46.415 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":10,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:37:45.094491+0000","last_up_change":"0.000000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:37:46.416 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.413+0000 7f5d2259f700 1 -- 172.21.15.12:0/1589122987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d1c0ccd40 msgr2=0x7f5d1c144110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:46.416 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.413+0000 7f5d2259f700 1 --2- 172.21.15.12:0/1589122987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d1c0ccd40 0x7f5d1c144110 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000c700 tx=0x7f5d1000ca10 comp rx=0 tx=0).stop 2023-07-08T12:37:46.416 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.413+0000 7f5d2259f700 1 -- 172.21.15.12:0/1589122987 shutdown_connections 2023-07-08T12:37:46.416 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.413+0000 7f5d2259f700 1 --2- 172.21.15.12:0/1589122987 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d1c0cd190 0x7f5d1c0c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:46.417 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.413+0000 7f5d2259f700 1 --2- 172.21.15.12:0/1589122987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d1c06cbd0 0x7f5d1c0c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:46.417 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.413+0000 7f5d2259f700 1 --2- 172.21.15.12:0/1589122987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d1c0ccd40 0x7f5d1c144110 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:46.417 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.413+0000 7f5d2259f700 1 -- 172.21.15.12:0/1589122987 >> 172.21.15.12:0/1589122987 conn(0x7f5d1c0b1920 msgr2=0x7f5d1c158710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:46.417 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.413+0000 7f5d2259f700 1 -- 172.21.15.12:0/1589122987 shutdown_connections 2023-07-08T12:37:46.417 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:46.413+0000 7f5d2259f700 1 -- 172.21.15.12:0/1589122987 wait complete. 2023-07-08T12:37:46.426 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:46.423+0000 7fc391c1f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-08T12:37:46.427 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-08T12:37:46.521 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:46.519+0000 7f6f35d8c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-08T12:37:46.645 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:46.642+0000 7fc391c1f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-08T12:37:46.748 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:46.746+0000 7f6f35d8c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-08T12:37:46.799 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:46.796+0000 7fc391c1f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-08T12:37:46.909 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:46.907+0000 7f6f35d8c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-08T12:37:46.915 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:46.913+0000 7fc391c1f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-08T12:37:47.030 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:47.028+0000 7f6f35d8c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-08T12:37:47.063 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:47.060+0000 7fc391c1f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-08T12:37:47.185 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:47.183+0000 7f6f35d8c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-08T12:37:47.341 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:47.338+0000 7fc391c1f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-08T12:37:47.471 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:47.469+0000 7f6f35d8c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-08T12:37:47.525 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:47.523+0000 7fc391c1f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-08T12:37:47.663 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:47.661+0000 7f6f35d8c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-08T12:37:48.116 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:48.112+0000 7fc391c1f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-08T12:37:48.292 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-07-08T12:37:48.290+0000 7f6f35d8c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-08T12:37:49.142 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:49.139+0000 7fc349e86700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-08T12:37:49.142 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-07-08T12:37:49.140+0000 7fc349e86700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-08T12:37:49.307 INFO:tasks.ceph.osd.6.smithi160.stderr:2023-07-08T12:37:49.305+0000 7fb3cb5ab640 -1 osd.6 0 log_to_monitors true 2023-07-08T12:37:49.308 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-07-08T12:37:49.305+0000 7f8404f78640 -1 osd.1 0 log_to_monitors true 2023-07-08T12:37:49.311 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-07-08T12:37:49.309+0000 7fcf3b222640 -1 osd.0 0 log_to_monitors true 2023-07-08T12:37:49.313 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-07-08T12:37:49.311+0000 7fe6cc782640 -1 osd.2 0 log_to_monitors true 2023-07-08T12:37:49.318 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-07-08T12:37:49.316+0000 7f58a5c98640 -1 osd.3 0 log_to_monitors true 2023-07-08T12:37:49.342 INFO:tasks.ceph.osd.4.smithi160.stderr:2023-07-08T12:37:49.341+0000 7efd7a32f640 -1 osd.4 0 log_to_monitors true 2023-07-08T12:37:49.346 INFO:tasks.ceph.osd.7.smithi160.stderr:2023-07-08T12:37:49.344+0000 7f0f5ad95640 -1 osd.7 0 log_to_monitors true 2023-07-08T12:37:49.351 INFO:tasks.ceph.osd.5.smithi160.stderr:2023-07-08T12:37:49.350+0000 7f3d40c65640 -1 osd.5 0 log_to_monitors true 2023-07-08T12:37:53.234 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-08T12:37:53.290 INFO:tasks.ceph.osd.6.smithi160.stderr:2023-07-08T12:37:53.288+0000 7fb3bf963700 -1 osd.6 0 waiting for initial osdmap 2023-07-08T12:37:53.302 INFO:tasks.ceph.osd.6.smithi160.stderr:2023-07-08T12:37:53.301+0000 7fb3b753d700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-08T12:37:53.303 INFO:tasks.ceph.osd.7.smithi160.stderr:2023-07-08T12:37:53.301+0000 7f0f4f94e700 -1 osd.7 0 waiting for initial osdmap 2023-07-08T12:37:53.315 INFO:tasks.ceph.osd.7.smithi160.stderr:2023-07-08T12:37:53.314+0000 7f0f46d27700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-08T12:37:53.320 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-07-08T12:37:53.318+0000 7fe6c0b3a700 -1 osd.2 0 waiting for initial osdmap 2023-07-08T12:37:53.324 INFO:tasks.ceph.osd.5.smithi160.stderr:2023-07-08T12:37:53.321+0000 7f3d3501d700 -1 osd.5 0 waiting for initial osdmap 2023-07-08T12:37:53.330 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-07-08T12:37:53.328+0000 7f83f9330700 -1 osd.1 0 waiting for initial osdmap 2023-07-08T12:37:53.332 INFO:tasks.ceph.osd.4.smithi160.stderr:2023-07-08T12:37:53.330+0000 7efd6e6e7700 -1 osd.4 0 waiting for initial osdmap 2023-07-08T12:37:53.332 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-07-08T12:37:53.330+0000 7f589a050700 -1 osd.3 0 waiting for initial osdmap 2023-07-08T12:37:53.336 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-07-08T12:37:53.334+0000 7fe6b8714700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-08T12:37:53.339 INFO:tasks.ceph.osd.5.smithi160.stderr:2023-07-08T12:37:53.338+0000 7f3d2cbf7700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-08T12:37:53.346 INFO:tasks.ceph.osd.4.smithi160.stderr:2023-07-08T12:37:53.344+0000 7efd662c1700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-08T12:37:53.348 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-07-08T12:37:53.346+0000 7f83f0f0a700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-08T12:37:53.349 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-07-08T12:37:53.346+0000 7f5891c2a700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-08T12:37:53.389 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-07-08T12:37:53.387+0000 7fcf2f5da700 -1 osd.0 0 waiting for initial osdmap 2023-07-08T12:37:53.407 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-07-08T12:37:53.405+0000 7fcf271b4700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-08T12:37:53.448 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.446+0000 7f1bee4ad700 1 Processor -- start 2023-07-08T12:37:53.448 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.446+0000 7f1bee4ad700 1 -- start start 2023-07-08T12:37:53.450 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1bee4ad700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1be80cf8f0 0x7f1be815af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:53.450 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1bee4ad700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be815b800 con 0x7f1be806cbd0 2023-07-08T12:37:53.450 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1bee4ad700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be80c98f0 con 0x7f1be80cb3a0 2023-07-08T12:37:53.450 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1bee4ad700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be80c9c90 con 0x7f1be80cf8f0 2023-07-08T12:37:53.451 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1be7fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f1be806cbd0 0x7f1be806cfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35992/0 (socket says 172.21.15.12:35992) 2023-07-08T12:37:53.451 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1be7fff700 1 -- 172.21.15.12:0/2539010195 learned_addr learned my addr 172.21.15.12:0/2539010195 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:37:53.451 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1be77fe700 1 --2- 172.21.15.12:0/2539010195 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1be80cf8f0 0x7f1be815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:53.451 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1be6ffd700 1 -- 172.21.15.12:0/2539010195 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928877853 0 0) 0x7f1be815b800 con 0x7f1be806cbd0 2023-07-08T12:37:53.451 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1be6ffd700 1 -- 172.21.15.12:0/2539010195 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bd00035b0 con 0x7f1be806cbd0 2023-07-08T12:37:53.451 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1be6ffd700 1 -- 172.21.15.12:0/2539010195 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024633866 0 0) 0x7f1be80c98f0 con 0x7f1be80cb3a0 2023-07-08T12:37:53.451 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1be6ffd700 1 -- 172.21.15.12:0/2539010195 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1be815b800 con 0x7f1be80cb3a0 2023-07-08T12:37:53.452 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1be6ffd700 1 -- 172.21.15.12:0/2539010195 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1309632622 0 0) 0x7f1bd00035b0 con 0x7f1be806cbd0 2023-07-08T12:37:53.452 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1be77fe700 1 -- 172.21.15.12:0/2539010195 >> v1:172.21.15.160:6789/0 conn(0x7f1be80cb3a0 legacy=0x7f1be8069fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:53.452 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1be77fe700 1 -- 172.21.15.12:0/2539010195 >> v1:172.21.15.12:6789/0 conn(0x7f1be806cbd0 legacy=0x7f1be806cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:53.452 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.447+0000 7f1be77fe700 1 -- 172.21.15.12:0/2539010195 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be815b520 con 0x7f1be80cf8f0 2023-07-08T12:37:53.452 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.448+0000 7f1be77fe700 1 --2- 172.21.15.12:0/2539010195 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1be80cf8f0 0x7f1be815af10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009870 tx=0x7f1bd8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc4d7e2cb5faa06a server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:53.452 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.448+0000 7f1be6ffd700 1 -- 172.21.15.12:0/2539010195 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc001670 con 0x7f1be80cf8f0 2023-07-08T12:37:53.452 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.448+0000 7f1be6ffd700 1 -- 172.21.15.12:0/2539010195 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd8004030 con 0x7f1be80cf8f0 2023-07-08T12:37:53.452 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.448+0000 7f1be6ffd700 1 -- 172.21.15.12:0/2539010195 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80043c0 con 0x7f1be80cf8f0 2023-07-08T12:37:53.455 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.448+0000 7f1bee4ad700 1 -- 172.21.15.12:0/2539010195 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1be80cf8f0 msgr2=0x7f1be815af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:53.455 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.448+0000 7f1bee4ad700 1 --2- 172.21.15.12:0/2539010195 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1be80cf8f0 0x7f1be815af10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009870 tx=0x7f1bd8009b80 comp rx=0 tx=0).stop 2023-07-08T12:37:53.455 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.448+0000 7f1bee4ad700 1 -- 172.21.15.12:0/2539010195 shutdown_connections 2023-07-08T12:37:53.455 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.448+0000 7f1bee4ad700 1 --2- 172.21.15.12:0/2539010195 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1be80cf8f0 0x7f1be815af10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:53.455 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.448+0000 7f1bee4ad700 1 -- 172.21.15.12:0/2539010195 >> 172.21.15.12:0/2539010195 conn(0x7f1be80b1920 msgr2=0x7f1be80cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:53.455 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.448+0000 7f1bee4ad700 1 -- 172.21.15.12:0/2539010195 shutdown_connections 2023-07-08T12:37:53.456 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.448+0000 7f1bee4ad700 1 -- 172.21.15.12:0/2539010195 wait complete. 2023-07-08T12:37:53.456 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.450+0000 7f1bee4ad700 1 Processor -- start 2023-07-08T12:37:53.456 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.450+0000 7f1bee4ad700 1 -- start start 2023-07-08T12:37:53.456 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1bee4ad700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1be806cbd0 0x7f1be8154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:53.456 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1bee4ad700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be80cb3a0 0x7f1be80c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:53.456 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1bee4ad700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1be80cf8f0 0x7f1be814e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:53.457 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1bee4ad700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be80c6780 con 0x7f1be80cb3a0 2023-07-08T12:37:53.457 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1bee4ad700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be80c69b0 con 0x7f1be80cf8f0 2023-07-08T12:37:53.457 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1bee4ad700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be80c6c90 con 0x7f1be806cbd0 2023-07-08T12:37:53.457 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1be77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be80cb3a0 0x7f1be80c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:53.457 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1be77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be80cb3a0 0x7f1be80c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48604/0 (socket says 172.21.15.12:48604) 2023-07-08T12:37:53.457 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1be77fe700 1 -- 172.21.15.12:0/4017887433 learned_addr learned my addr 172.21.15.12:0/4017887433 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:37:53.457 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1be7fff700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1be806cbd0 0x7f1be8154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:53.457 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1beca4a700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1be80cf8f0 0x7f1be814e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:53.458 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1be77fe700 1 -- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1be806cbd0 msgr2=0x7f1be8154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:53.458 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1be77fe700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1be806cbd0 0x7f1be8154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:53.458 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1be77fe700 1 -- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1be80cf8f0 msgr2=0x7f1be814e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:53.458 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1be77fe700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1be80cf8f0 0x7f1be814e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:53.458 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1be77fe700 1 -- 172.21.15.12:0/4017887433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd80094a0 con 0x7f1be80cb3a0 2023-07-08T12:37:53.458 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1be77fe700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be80cb3a0 0x7f1be80c6240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80090b0 tx=0x7f1bd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:53.458 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1be4ff9700 1 -- 172.21.15.12:0/4017887433 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8003e60 con 0x7f1be80cb3a0 2023-07-08T12:37:53.459 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1be4ff9700 1 -- 172.21.15.12:0/4017887433 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd801f070 con 0x7f1be80cb3a0 2023-07-08T12:37:53.459 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1bee4ad700 1 -- 172.21.15.12:0/4017887433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be80c6f70 con 0x7f1be80cb3a0 2023-07-08T12:37:53.459 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.451+0000 7f1be4ff9700 1 -- 172.21.15.12:0/4017887433 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8011850 con 0x7f1be80cb3a0 2023-07-08T12:37:53.459 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.452+0000 7f1bee4ad700 1 -- 172.21.15.12:0/4017887433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be814eab0 con 0x7f1be80cb3a0 2023-07-08T12:37:53.459 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.452+0000 7f1bce7fc700 1 -- 172.21.15.12:0/4017887433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc00052f0 con 0x7f1be80cb3a0 2023-07-08T12:37:53.460 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.453+0000 7f1be4ff9700 1 -- 172.21.15.12:0/4017887433 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f1bd80119b0 con 0x7f1be80cb3a0 2023-07-08T12:37:53.461 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.453+0000 7f1be4ff9700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1bd406ac50 0x7f1bd406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:37:53.461 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.453+0000 7f1be4ff9700 1 -- 172.21.15.12:0/4017887433 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f1bd8021070 con 0x7f1be80cb3a0 2023-07-08T12:37:53.461 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.453+0000 7f1be7fff700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1bd406ac50 0x7f1bd406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:37:53.461 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.454+0000 7f1be7fff700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1bd406ac50 0x7f1bd406d100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc000f60 tx=0x7f1bdc001670 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:37:53.462 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.458+0000 7f1be4ff9700 1 -- 172.21.15.12:0/4017887433 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1bd8055d00 con 0x7f1be80cb3a0 2023-07-08T12:37:53.658 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.655+0000 7f1bce7fc700 1 -- 172.21.15.12:0/4017887433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bc00055a0 con 0x7f1be80cb3a0 2023-07-08T12:37:53.659 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.656+0000 7f1be4ff9700 1 -- 172.21.15.12:0/4017887433 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f1bd8059320 con 0x7f1be80cb3a0 2023-07-08T12:37:53.659 INFO:teuthology.misc.health.smithi012.stdout: 2023-07-08T12:37:53.659 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":12,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:37:51.259672+0000","last_up_change":"0.000000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:37:53.661 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.658+0000 7f1bce7fc700 1 -- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1bd406ac50 msgr2=0x7f1bd406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:53.661 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.658+0000 7f1bce7fc700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1bd406ac50 0x7f1bd406d100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc000f60 tx=0x7f1bdc001670 comp rx=0 tx=0).stop 2023-07-08T12:37:53.662 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.658+0000 7f1bce7fc700 1 -- 172.21.15.12:0/4017887433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be80cb3a0 msgr2=0x7f1be80c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:37:53.662 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.658+0000 7f1bce7fc700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be80cb3a0 0x7f1be80c6240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80090b0 tx=0x7f1bd8004750 comp rx=0 tx=0).stop 2023-07-08T12:37:53.662 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.658+0000 7f1bce7fc700 1 -- 172.21.15.12:0/4017887433 shutdown_connections 2023-07-08T12:37:53.662 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.658+0000 7f1bce7fc700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1bd406ac50 0x7f1bd406d100 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:53.662 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.658+0000 7f1bce7fc700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1be80cf8f0 0x7f1be814e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:53.662 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.658+0000 7f1bce7fc700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1be806cbd0 0x7f1be8154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:53.662 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.658+0000 7f1bce7fc700 1 --2- 172.21.15.12:0/4017887433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be80cb3a0 0x7f1be80c6240 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:37:53.663 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.658+0000 7f1bce7fc700 1 -- 172.21.15.12:0/4017887433 >> 172.21.15.12:0/4017887433 conn(0x7f1be80b1920 msgr2=0x7f1be80719b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:37:53.663 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.658+0000 7f1bce7fc700 1 -- 172.21.15.12:0/4017887433 shutdown_connections 2023-07-08T12:37:53.663 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:37:53.658+0000 7f1bce7fc700 1 -- 172.21.15.12:0/4017887433 wait complete. 2023-07-08T12:37:53.672 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-08T12:38:00.478 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:00.605 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.599+0000 7f97dc75b700 1 Processor -- start 2023-07-08T12:38:00.605 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.600+0000 7f97dc75b700 1 -- start start 2023-07-08T12:38:00.606 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.600+0000 7f97dc75b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97d406cc80 0x7f97d406bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.606 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97dc75b700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d406c740 con 0x7f97d415c240 2023-07-08T12:38:00.606 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97dc75b700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d40ce540 con 0x7f97d406c300 2023-07-08T12:38:00.606 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97dc75b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40d02a0 con 0x7f97d406cc80 2023-07-08T12:38:00.606 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97da4f7700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f97d415c240 0x7f97d40caf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36186/0 (socket says 172.21.15.12:36186) 2023-07-08T12:38:00.606 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97da4f7700 1 -- 172.21.15.12:0/2300936182 learned_addr learned my addr 172.21.15.12:0/2300936182 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:00.607 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d9cf6700 1 --2- 172.21.15.12:0/2300936182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97d406cc80 0x7f97d406bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.607 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020949447 0 0) 0x7f97d0001110 con 0x7f97d415c240 2023-07-08T12:38:00.607 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97c00035b0 con 0x7f97d415c240 2023-07-08T12:38:00.607 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2985978926 0 0) 0x7f97c00035b0 con 0x7f97d415c240 2023-07-08T12:38:00.607 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97d406cc80 msgr2=0x7f97d406bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:00.607 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 --2- 172.21.15.12:0/2300936182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97d406cc80 0x7f97d406bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.607 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 >> v1:172.21.15.160:6789/0 conn(0x7f97d406c300 legacy=0x7f97d415fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:00.608 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d40ce7b0 con 0x7f97d415c240 2023-07-08T12:38:00.608 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f97d0001800 con 0x7f97d415c240 2023-07-08T12:38:00.608 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 >> v1:172.21.15.12:6789/0 conn(0x7f97d415c240 legacy=0x7f97d40caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:00.608 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 --2- 172.21.15.12:0/2300936182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97c0003a30 0x7f97c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.608 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 --2- 172.21.15.12:0/2300936182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97d406cc80 0x7f97c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.608 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 --2- 172.21.15.12:0/2300936182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c0004d90 0x7f97c0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.608 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40d02a0 con 0x7f97c0003a30 2023-07-08T12:38:00.609 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c000bac0 con 0x7f97c0004d90 2023-07-08T12:38:00.609 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c000bc80 con 0x7f97d406cc80 2023-07-08T12:38:00.609 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.601+0000 7f97da4f7700 1 --2- 172.21.15.12:0/2300936182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97c0003a30 0x7f97c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.609 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.602+0000 7f97d9cf6700 1 --2- 172.21.15.12:0/2300936182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97d406cc80 0x7f97c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.609 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.602+0000 7f97dacf8700 1 --2- 172.21.15.12:0/2300936182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c0004d90 0x7f97c0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.609 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.602+0000 7f97da4f7700 1 -- 172.21.15.12:0/2300936182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97d406cc80 msgr2=0x7f97c0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:00.609 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.602+0000 7f97da4f7700 1 --2- 172.21.15.12:0/2300936182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97d406cc80 0x7f97c0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.609 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.602+0000 7f97da4f7700 1 -- 172.21.15.12:0/2300936182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c0004d90 msgr2=0x7f97c0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:00.610 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.602+0000 7f97da4f7700 1 --2- 172.21.15.12:0/2300936182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c0004d90 0x7f97c0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.610 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.602+0000 7f97da4f7700 1 -- 172.21.15.12:0/2300936182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c00035b0 con 0x7f97c0003a30 2023-07-08T12:38:00.610 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.602+0000 7f97da4f7700 1 --2- 172.21.15.12:0/2300936182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97c0003a30 0x7f97c0003e00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f97d000c1c0 tx=0x7f97c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38de8522ce7422c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:00.610 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.602+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d000cd60 con 0x7f97c0003a30 2023-07-08T12:38:00.610 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.602+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d0005960 con 0x7f97c0003a30 2023-07-08T12:38:00.610 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.602+0000 7f97d94f5700 1 -- 172.21.15.12:0/2300936182 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0005cf0 con 0x7f97c0003a30 2023-07-08T12:38:00.611 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.603+0000 7f97dc75b700 1 -- 172.21.15.12:0/2300936182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97c0003a30 msgr2=0x7f97c0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:00.611 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.603+0000 7f97dc75b700 1 --2- 172.21.15.12:0/2300936182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97c0003a30 0x7f97c0003e00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f97d000c1c0 tx=0x7f97c0005880 comp rx=0 tx=0).stop 2023-07-08T12:38:00.611 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.603+0000 7f97dacf8700 1 -- 172.21.15.12:0/2300936182 reap_dead start 2023-07-08T12:38:00.611 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.603+0000 7f97dc75b700 1 -- 172.21.15.12:0/2300936182 shutdown_connections 2023-07-08T12:38:00.611 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.603+0000 7f97dc75b700 1 -- 172.21.15.12:0/2300936182 >> 172.21.15.12:0/2300936182 conn(0x7f97d406e590 msgr2=0x7f97d406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:00.611 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.603+0000 7f97dc75b700 1 -- 172.21.15.12:0/2300936182 shutdown_connections 2023-07-08T12:38:00.611 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.603+0000 7f97dc75b700 1 -- 172.21.15.12:0/2300936182 wait complete. 2023-07-08T12:38:00.614 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.604+0000 7f97dc75b700 1 Processor -- start 2023-07-08T12:38:00.614 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.604+0000 7f97dc75b700 1 -- start start 2023-07-08T12:38:00.614 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.605+0000 7f97dc75b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97c0003a30 0x7f97d4152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.614 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.605+0000 7f97dc75b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d40cd160 0x7f97d41528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.614 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.605+0000 7f97dc75b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d40b40f0 0x7f97d415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.614 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.605+0000 7f97dc75b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40cd5d0 con 0x7f97d40b40f0 2023-07-08T12:38:00.615 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.605+0000 7f97dc75b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40b48d0 con 0x7f97d40cd160 2023-07-08T12:38:00.615 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.605+0000 7f97dc75b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40b4bb0 con 0x7f97c0003a30 2023-07-08T12:38:00.615 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.605+0000 7f97dacf8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d40b40f0 0x7f97d415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.615 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97dacf8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d40b40f0 0x7f97d415e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48806/0 (socket says 172.21.15.12:48806) 2023-07-08T12:38:00.615 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97dacf8700 1 -- 172.21.15.12:0/3423661011 learned_addr learned my addr 172.21.15.12:0/3423661011 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:00.615 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97da4f7700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97c0003a30 0x7f97d4152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.615 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97d9cf6700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d40cd160 0x7f97d41528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.616 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97dacf8700 1 -- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97c0003a30 msgr2=0x7f97d4152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:00.616 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97dacf8700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97c0003a30 0x7f97d4152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.616 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97dacf8700 1 -- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d40cd160 msgr2=0x7f97d41528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:00.616 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97dacf8700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d40cd160 0x7f97d41528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.616 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97dacf8700 1 -- 172.21.15.12:0/3423661011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d40b4e90 con 0x7f97d40b40f0 2023-07-08T12:38:00.616 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97da4f7700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97c0003a30 0x7f97d4152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:00.616 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97d9cf6700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d40cd160 0x7f97d41528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:00.616 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97dacf8700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d40b40f0 0x7f97d415e980 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f97d40b3450 tx=0x7f97cc0021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:00.617 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97c6ffd700 1 -- 172.21.15.12:0/3423661011 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc003c10 con 0x7f97d40b40f0 2023-07-08T12:38:00.617 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97c6ffd700 1 -- 172.21.15.12:0/3423661011 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc003d70 con 0x7f97d40b40f0 2023-07-08T12:38:00.617 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97dc75b700 1 -- 172.21.15.12:0/3423661011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d40b5170 con 0x7f97d40b40f0 2023-07-08T12:38:00.617 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97c6ffd700 1 -- 172.21.15.12:0/3423661011 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc00c990 con 0x7f97d40b40f0 2023-07-08T12:38:00.617 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.606+0000 7f97dc75b700 1 -- 172.21.15.12:0/3423661011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d40b4560 con 0x7f97d40b40f0 2023-07-08T12:38:00.617 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.607+0000 7f97dc75b700 1 -- 172.21.15.12:0/3423661011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d40b5690 con 0x7f97d40b40f0 2023-07-08T12:38:00.617 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.608+0000 7f97c6ffd700 1 -- 172.21.15.12:0/3423661011 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f97cc00caf0 con 0x7f97d40b40f0 2023-07-08T12:38:00.618 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.608+0000 7f97c6ffd700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f97b806ace0 0x7f97b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.618 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.608+0000 7f97c6ffd700 1 -- 172.21.15.12:0/3423661011 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f97cc089150 con 0x7f97d40b40f0 2023-07-08T12:38:00.618 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.608+0000 7f97da4f7700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f97b806ace0 0x7f97b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.618 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.609+0000 7f97da4f7700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f97b806ace0 0x7f97b806d190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97d0015040 tx=0x7f97d0007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:00.618 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.613+0000 7f97c6ffd700 1 -- 172.21.15.12:0/3423661011 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f97d40b5690 con 0x7f97d40b40f0 2023-07-08T12:38:00.812 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.810+0000 7f97dc75b700 1 -- 172.21.15.12:0/3423661011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97d40b5690 con 0x7f97d40b40f0 2023-07-08T12:38:00.812 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.811+0000 7f97c6ffd700 1 -- 172.21.15.12:0/3423661011 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12838 (secure 0 0 0) 0x7f97d40b5690 con 0x7f97d40b40f0 2023-07-08T12:38:00.813 INFO:teuthology.misc.health.smithi012.stdout: 2023-07-08T12:38:00.813 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":17,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:37:58.345840+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:00.815 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.813+0000 7f97dc75b700 1 -- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f97b806ace0 msgr2=0x7f97b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:00.815 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.813+0000 7f97dc75b700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f97b806ace0 0x7f97b806d190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97d0015040 tx=0x7f97d0007000 comp rx=0 tx=0).stop 2023-07-08T12:38:00.816 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.813+0000 7f97dc75b700 1 -- 172.21.15.12:0/3423661011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d40b40f0 msgr2=0x7f97d415e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:00.816 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.813+0000 7f97dc75b700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d40b40f0 0x7f97d415e980 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f97d40b3450 tx=0x7f97cc0021c0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.816 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.813+0000 7f97dc75b700 1 -- 172.21.15.12:0/3423661011 shutdown_connections 2023-07-08T12:38:00.817 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.813+0000 7f97dc75b700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f97b806ace0 0x7f97b806d190 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.817 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.813+0000 7f97dc75b700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97d40b40f0 0x7f97d415e980 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.817 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.813+0000 7f97dc75b700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97c0003a30 0x7f97d4152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.818 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.813+0000 7f97dc75b700 1 --2- 172.21.15.12:0/3423661011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d40cd160 0x7f97d41528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.818 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.813+0000 7f97dc75b700 1 -- 172.21.15.12:0/3423661011 >> 172.21.15.12:0/3423661011 conn(0x7f97d406e590 msgr2=0x7f97d40b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:00.818 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.813+0000 7f97dc75b700 1 -- 172.21.15.12:0/3423661011 shutdown_connections 2023-07-08T12:38:00.818 INFO:teuthology.misc.health.smithi012.stderr:2023-07-08T12:38:00.813+0000 7f97dc75b700 1 -- 172.21.15.12:0/3423661011 wait complete. 2023-07-08T12:38:00.826 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-07-08T12:38:00.826 INFO:tasks.ceph:Creating RBD pool 2023-07-08T12:38:00.827 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd pool create rbd 8 2023-07-08T12:38:00.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.948+0000 7effed0d5700 1 Processor -- start 2023-07-08T12:38:00.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.948+0000 7effed0d5700 1 -- start start 2023-07-08T12:38:00.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.949+0000 7effed0d5700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7effe806e0c0 0x7effe81608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.949+0000 7effed0d5700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7effe80cc240 0x7effe80cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.949+0000 7effed0d5700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe80ccbe0 con 0x7effe8156a10 2023-07-08T12:38:00.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.949+0000 7effed0d5700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe8160410 con 0x7effe806e0c0 2023-07-08T12:38:00.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.949+0000 7effed0d5700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe8156de0 con 0x7effe80cc240 2023-07-08T12:38:00.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.949+0000 7effe659c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7effe806e0c0 0x7effe81608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.949+0000 7effe659c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7effe806e0c0 0x7effe81608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48812/0 (socket says 172.21.15.12:48812) 2023-07-08T12:38:00.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.949+0000 7effe659c700 1 -- 172.21.15.12:0/905120755 learned_addr learned my addr 172.21.15.12:0/905120755 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:00.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.949+0000 7effe759e700 1 --2- 172.21.15.12:0/905120755 >> v2:172.21.15.160:3300/0 conn(0x7effe80cc240 0x7effe80cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.949+0000 7effe659c700 1 -- 172.21.15.12:0/905120755 >> v2:172.21.15.160:3300/0 conn(0x7effe80cc240 msgr2=0x7effe80cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:00.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.949+0000 7effe659c700 1 --2- 172.21.15.12:0/905120755 >> v2:172.21.15.160:3300/0 conn(0x7effe80cc240 0x7effe80cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.949+0000 7effe659c700 1 -- 172.21.15.12:0/905120755 >> v1:172.21.15.160:6789/0 conn(0x7effe8156a10 legacy=0x7effe815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:00.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.949+0000 7effe659c700 1 -- 172.21.15.12:0/905120755 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe806ec70 con 0x7effe806e0c0 2023-07-08T12:38:00.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.950+0000 7effe659c700 1 --2- 172.21.15.12:0/905120755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe806e0c0 0x7effe81608a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7effe806c880 tx=0x7effd8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66afd5822916ef27 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:00.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.950+0000 7effe5d9b700 1 -- 172.21.15.12:0/905120755 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8011830 con 0x7effe806e0c0 2023-07-08T12:38:00.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.950+0000 7effe5d9b700 1 -- 172.21.15.12:0/905120755 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd8011990 con 0x7effe806e0c0 2023-07-08T12:38:00.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.950+0000 7effe5d9b700 1 -- 172.21.15.12:0/905120755 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8011d20 con 0x7effe806e0c0 2023-07-08T12:38:00.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.950+0000 7effed0d5700 1 -- 172.21.15.12:0/905120755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe806e0c0 msgr2=0x7effe81608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:00.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.950+0000 7effed0d5700 1 --2- 172.21.15.12:0/905120755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe806e0c0 0x7effe81608a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7effe806c880 tx=0x7effd8009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.950+0000 7effed0d5700 1 -- 172.21.15.12:0/905120755 shutdown_connections 2023-07-08T12:38:00.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.950+0000 7effed0d5700 1 --2- 172.21.15.12:0/905120755 >> v2:172.21.15.160:3300/0 conn(0x7effe80cc240 0x7effe80cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.950+0000 7effed0d5700 1 --2- 172.21.15.12:0/905120755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe806e0c0 0x7effe81608a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.950+0000 7effed0d5700 1 -- 172.21.15.12:0/905120755 >> 172.21.15.12:0/905120755 conn(0x7effe8098b50 msgr2=0x7effe806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:00.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.950+0000 7effed0d5700 1 -- 172.21.15.12:0/905120755 shutdown_connections 2023-07-08T12:38:00.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.950+0000 7effed0d5700 1 -- 172.21.15.12:0/905120755 wait complete. 2023-07-08T12:38:00.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.952+0000 7effed0d5700 1 Processor -- start 2023-07-08T12:38:00.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.952+0000 7effed0d5700 1 -- start start 2023-07-08T12:38:00.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effed0d5700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effe806e0c0 0x7effe80b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effed0d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe80cc240 0x7effe8151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effed0d5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effe8156a10 0x7effe80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effed0d5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe80ac7c0 con 0x7effe80cc240 2023-07-08T12:38:00.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effed0d5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe80ad020 con 0x7effe8156a10 2023-07-08T12:38:00.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effed0d5700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe8152140 con 0x7effe806e0c0 2023-07-08T12:38:00.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effe659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe80cc240 0x7effe8151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effe659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe80cc240 0x7effe8151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48818/0 (socket says 172.21.15.12:48818) 2023-07-08T12:38:00.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effe659c700 1 -- 172.21.15.12:0/324556754 learned_addr learned my addr 172.21.15.12:0/324556754 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:00.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effe759e700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effe8156a10 0x7effe80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effe6d9d700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effe806e0c0 0x7effe80b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effe659c700 1 -- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effe806e0c0 msgr2=0x7effe80b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:00.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effe659c700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effe806e0c0 0x7effe80b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effe659c700 1 -- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effe8156a10 msgr2=0x7effe80b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:00.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effe659c700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effe8156a10 0x7effe80b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:00.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.953+0000 7effe659c700 1 -- 172.21.15.12:0/324556754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd8009550 con 0x7effe80cc240 2023-07-08T12:38:00.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.954+0000 7effe659c700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe80cc240 0x7effe8151910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7effd8000c00 tx=0x7effd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:00.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.954+0000 7effcf7fe700 1 -- 172.21.15.12:0/324556754 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8004570 con 0x7effe80cc240 2023-07-08T12:38:00.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.954+0000 7effcf7fe700 1 -- 172.21.15.12:0/324556754 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd801f070 con 0x7effe80cc240 2023-07-08T12:38:00.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.954+0000 7effcf7fe700 1 -- 172.21.15.12:0/324556754 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd80038c0 con 0x7effe80cc240 2023-07-08T12:38:00.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.954+0000 7effed0d5700 1 -- 172.21.15.12:0/324556754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe80ac9d0 con 0x7effe80cc240 2023-07-08T12:38:00.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.954+0000 7effed0d5700 1 -- 172.21.15.12:0/324556754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe8151e50 con 0x7effe80cc240 2023-07-08T12:38:00.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.955+0000 7effed0d5700 1 -- 172.21.15.12:0/324556754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe804ea60 con 0x7effe80cc240 2023-07-08T12:38:00.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.955+0000 7effcf7fe700 1 -- 172.21.15.12:0/324556754 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7effd8021030 con 0x7effe80cc240 2023-07-08T12:38:00.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.955+0000 7effcf7fe700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7effd406ace0 0x7effd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:00.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.956+0000 7effcf7fe700 1 -- 172.21.15.12:0/324556754 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7effd808a460 con 0x7effe80cc240 2023-07-08T12:38:00.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.956+0000 7effe6d9d700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7effd406ace0 0x7effd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:00.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.956+0000 7effe6d9d700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7effd406ace0 0x7effd406d190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effdc000fb0 tx=0x7effdc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:00.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:00.960+0000 7effcf7fe700 1 -- 172.21.15.12:0/324556754 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7effd8055b70 con 0x7effe80cc240 2023-07-08T12:38:01.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.156+0000 7effed0d5700 1 -- 172.21.15.12:0/324556754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7effe80c61b0 con 0x7effe80cc240 2023-07-08T12:38:01.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.365+0000 7effcf7fe700 1 -- 172.21.15.12:0/324556754 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v18) v1 ==== 109+0+0 (secure 0 0 0) 0x7effd8059190 con 0x7effe80cc240 2023-07-08T12:38:01.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.368+0000 7effed0d5700 1 -- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7effd406ace0 msgr2=0x7effd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:01.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.368+0000 7effed0d5700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7effd406ace0 0x7effd406d190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effdc000fb0 tx=0x7effdc009040 comp rx=0 tx=0).stop 2023-07-08T12:38:01.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.368+0000 7effed0d5700 1 -- 172.21.15.12:0/324556754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe80cc240 msgr2=0x7effe8151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:01.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.368+0000 7effed0d5700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe80cc240 0x7effe8151910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7effd8000c00 tx=0x7effd8004750 comp rx=0 tx=0).stop 2023-07-08T12:38:01.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.368+0000 7effed0d5700 1 -- 172.21.15.12:0/324556754 shutdown_connections 2023-07-08T12:38:01.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.368+0000 7effed0d5700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7effd406ace0 0x7effd406d190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:01.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.368+0000 7effed0d5700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effe8156a10 0x7effe80b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:01.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.368+0000 7effed0d5700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effe806e0c0 0x7effe80b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:01.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.368+0000 7effed0d5700 1 --2- 172.21.15.12:0/324556754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe80cc240 0x7effe8151910 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:01.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.368+0000 7effed0d5700 1 -- 172.21.15.12:0/324556754 >> 172.21.15.12:0/324556754 conn(0x7effe8098b50 msgr2=0x7effe80b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:01.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.369+0000 7effed0d5700 1 -- 172.21.15.12:0/324556754 shutdown_connections 2023-07-08T12:38:01.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.369+0000 7effed0d5700 1 -- 172.21.15.12:0/324556754 wait complete. 2023-07-08T12:38:01.375 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'rbd' created 2023-07-08T12:38:01.383 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-07-08T12:38:01.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.508+0000 7f3b5c26b700 1 Processor -- start 2023-07-08T12:38:01.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.508+0000 7f3b5c26b700 1 -- start start 2023-07-08T12:38:01.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b5c26b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3b5415d8b0 0x7f3b5415e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b5c26b700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3b54069f30 0x7f3b5406a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b5c26b700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b540ccbe0 con 0x7f3b540cf8a0 2023-07-08T12:38:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b5c26b700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b540725e0 con 0x7f3b5415d8b0 2023-07-08T12:38:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b5c26b700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5406a8e0 con 0x7f3b54069f30 2023-07-08T12:38:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b5a007700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3b5415d8b0 0x7f3b5415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b5a808700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3b540cf8a0 0x7f3b540cfd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36224/0 (socket says 172.21.15.12:36224) 2023-07-08T12:38:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b5a808700 1 -- 172.21.15.12:0/873426179 learned_addr learned my addr 172.21.15.12:0/873426179 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b59806700 1 --2- 172.21.15.12:0/873426179 >> v2:172.21.15.160:3300/0 conn(0x7f3b54069f30 0x7f3b5406a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b5a007700 1 -- 172.21.15.12:0/873426179 >> v2:172.21.15.160:3300/0 conn(0x7f3b54069f30 msgr2=0x7f3b5406a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b5a007700 1 --2- 172.21.15.12:0/873426179 >> v2:172.21.15.160:3300/0 conn(0x7f3b54069f30 0x7f3b5406a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b5a007700 1 -- 172.21.15.12:0/873426179 >> v1:172.21.15.12:6789/0 conn(0x7f3b540cf8a0 legacy=0x7f3b540cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b5a007700 1 -- 172.21.15.12:0/873426179 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5415ec50 con 0x7f3b5415d8b0 2023-07-08T12:38:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.509+0000 7f3b59005700 1 -- 172.21.15.12:0/873426179 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656681442 0 0) 0x7f3b4c001490 con 0x7f3b540cf8a0 2023-07-08T12:38:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.510+0000 7f3b5a007700 1 --2- 172.21.15.12:0/873426179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b5415d8b0 0x7f3b5415e710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b5406e6c0 tx=0x7f3b44009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0aab2a16ac4102f server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.510+0000 7f3b59005700 1 -- 172.21.15.12:0/873426179 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44011830 con 0x7f3b5415d8b0 2023-07-08T12:38:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.510+0000 7f3b59005700 1 -- 172.21.15.12:0/873426179 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b44011990 con 0x7f3b5415d8b0 2023-07-08T12:38:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.510+0000 7f3b59005700 1 -- 172.21.15.12:0/873426179 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44011d20 con 0x7f3b5415d8b0 2023-07-08T12:38:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.510+0000 7f3b5c26b700 1 -- 172.21.15.12:0/873426179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b5415d8b0 msgr2=0x7f3b5415e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.510+0000 7f3b5c26b700 1 --2- 172.21.15.12:0/873426179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b5415d8b0 0x7f3b5415e710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b5406e6c0 tx=0x7f3b44009b20 comp rx=0 tx=0).stop 2023-07-08T12:38:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.510+0000 7f3b5c26b700 1 -- 172.21.15.12:0/873426179 shutdown_connections 2023-07-08T12:38:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.510+0000 7f3b5c26b700 1 --2- 172.21.15.12:0/873426179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b5415d8b0 0x7f3b5415e710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.510+0000 7f3b5c26b700 1 --2- 172.21.15.12:0/873426179 >> v2:172.21.15.160:3300/0 conn(0x7f3b54069f30 0x7f3b5406a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.510+0000 7f3b5c26b700 1 -- 172.21.15.12:0/873426179 >> 172.21.15.12:0/873426179 conn(0x7f3b5406fd80 msgr2=0x7f3b540b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.510+0000 7f3b5c26b700 1 -- 172.21.15.12:0/873426179 shutdown_connections 2023-07-08T12:38:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.510+0000 7f3b5c26b700 1 -- 172.21.15.12:0/873426179 wait complete. 2023-07-08T12:38:01.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.512+0000 7f3b5c26b700 1 Processor -- start 2023-07-08T12:38:01.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.512+0000 7f3b5c26b700 1 -- start start 2023-07-08T12:38:01.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.512+0000 7f3b5c26b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54069f30 0x7f3b541f01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:01.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.512+0000 7f3b5c26b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b540cf8a0 0x7f3b541f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:01.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.512+0000 7f3b5c26b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b5415d8b0 0x7f3b541f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.512+0000 7f3b5c26b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b541595f0 con 0x7f3b5415d8b0 2023-07-08T12:38:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.512+0000 7f3b5c26b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b541f3cc0 con 0x7f3b54069f30 2023-07-08T12:38:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.512+0000 7f3b5c26b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b541f3f40 con 0x7f3b540cf8a0 2023-07-08T12:38:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.512+0000 7f3b5a808700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b5415d8b0 0x7f3b541f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b5a808700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b5415d8b0 0x7f3b541f2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48834/0 (socket says 172.21.15.12:48834) 2023-07-08T12:38:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b5a808700 1 -- 172.21.15.12:0/1805644077 learned_addr learned my addr 172.21.15.12:0/1805644077 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:01.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b59806700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b540cf8a0 0x7f3b541f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:01.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b5a007700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54069f30 0x7f3b541f01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:01.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b5a808700 1 -- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b540cf8a0 msgr2=0x7f3b541f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:01.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b5a808700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b540cf8a0 0x7f3b541f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b5a808700 1 -- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54069f30 msgr2=0x7f3b541f01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b5a808700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54069f30 0x7f3b541f01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b5a808700 1 -- 172.21.15.12:0/1805644077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b440094a0 con 0x7f3b5415d8b0 2023-07-08T12:38:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b59806700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b540cf8a0 0x7f3b541f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b5a808700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b5415d8b0 0x7f3b541f2ee0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00ccd0 tx=0x7f3b4c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b4affd700 1 -- 172.21.15.12:0/1805644077 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c00ab80 con 0x7f3b5415d8b0 2023-07-08T12:38:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b4affd700 1 -- 172.21.15.12:0/1805644077 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c00ace0 con 0x7f3b5415d8b0 2023-07-08T12:38:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b5c26b700 1 -- 172.21.15.12:0/1805644077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b541f4220 con 0x7f3b5415d8b0 2023-07-08T12:38:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b4affd700 1 -- 172.21.15.12:0/1805644077 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c007310 con 0x7f3b5415d8b0 2023-07-08T12:38:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.513+0000 7f3b5c26b700 1 -- 172.21.15.12:0/1805644077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b54159800 con 0x7f3b5415d8b0 2023-07-08T12:38:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.514+0000 7f3b5c26b700 1 -- 172.21.15.12:0/1805644077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b54159a20 con 0x7f3b5415d8b0 2023-07-08T12:38:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.515+0000 7f3b4affd700 1 -- 172.21.15.12:0/1805644077 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3b4c007470 con 0x7f3b5415d8b0 2023-07-08T12:38:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.515+0000 7f3b4affd700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3b3806ace0 0x7f3b3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.515+0000 7f3b4affd700 1 -- 172.21.15.12:0/1805644077 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f3b4c088d50 con 0x7f3b5415d8b0 2023-07-08T12:38:01.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.515+0000 7f3b5a007700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3b3806ace0 0x7f3b3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:01.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.516+0000 7f3b5a007700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3b3806ace0 0x7f3b3806d190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3b440090b0 tx=0x7f3b44008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:01.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.520+0000 7f3b4affd700 1 -- 172.21.15.12:0/1805644077 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3b4c08c030 con 0x7f3b5415d8b0 2023-07-08T12:38:01.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:01.720+0000 7f3b5c26b700 1 -- 172.21.15.12:0/1805644077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3b540ccbe0 con 0x7f3b5415d8b0 2023-07-08T12:38:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.377+0000 7f3b4affd700 1 -- 172.21.15.12:0/1805644077 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v19) v1 ==== 173+0+0 (secure 0 0 0) 0x7f3b540ccbe0 con 0x7f3b5415d8b0 2023-07-08T12:38:02.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.381+0000 7f3b5c26b700 1 -- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3b3806ace0 msgr2=0x7f3b3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.381+0000 7f3b5c26b700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3b3806ace0 0x7f3b3806d190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3b440090b0 tx=0x7f3b44008030 comp rx=0 tx=0).stop 2023-07-08T12:38:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.381+0000 7f3b5c26b700 1 -- 172.21.15.12:0/1805644077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b5415d8b0 msgr2=0x7f3b541f2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.381+0000 7f3b5c26b700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b5415d8b0 0x7f3b541f2ee0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00ccd0 tx=0x7f3b4c00a060 comp rx=0 tx=0).stop 2023-07-08T12:38:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.381+0000 7f3b5c26b700 1 -- 172.21.15.12:0/1805644077 shutdown_connections 2023-07-08T12:38:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.381+0000 7f3b5c26b700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3b3806ace0 0x7f3b3806d190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.381+0000 7f3b5c26b700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b5415d8b0 0x7f3b541f2ee0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.381+0000 7f3b5c26b700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b54069f30 0x7f3b541f01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.381+0000 7f3b5c26b700 1 --2- 172.21.15.12:0/1805644077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b540cf8a0 0x7f3b541f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.381+0000 7f3b5c26b700 1 -- 172.21.15.12:0/1805644077 >> 172.21.15.12:0/1805644077 conn(0x7f3b5406fd80 msgr2=0x7f3b540b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.381+0000 7f3b5c26b700 1 -- 172.21.15.12:0/1805644077 shutdown_connections 2023-07-08T12:38:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.381+0000 7f3b5c26b700 1 -- 172.21.15.12:0/1805644077 wait complete. 2023-07-08T12:38:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:enabled application 'rbd' on pool 'rbd' 2023-07-08T12:38:02.398 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-07-08T12:38:02.399 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-08T12:38:02.399 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-08T12:38:02.401 INFO:tasks.ceph.mds.a:Started 2023-07-08T12:38:02.402 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-08T12:38:02.402 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-08T12:38:02.404 INFO:tasks.ceph.mds.c:Started 2023-07-08T12:38:02.405 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-08T12:38:02.405 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-08T12:38:02.407 INFO:tasks.ceph.mds.b:Started 2023-07-08T12:38:02.408 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-08T12:38:02.408 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-08T12:38:02.410 INFO:tasks.ceph.mds.d:Started 2023-07-08T12:38:02.410 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-07-08T12:38:02.411 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:02.490 INFO:tasks.ceph.mds.c.smithi012.stdout:starting mds.c at 2023-07-08T12:38:02.491 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-07-08T12:38:02.492 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-07-08T12:38:02.492 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-07-08T12:38:02.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.547+0000 7f7e50e4f700 1 Processor -- start 2023-07-08T12:38:02.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.547+0000 7f7e50e4f700 1 -- start start 2023-07-08T12:38:02.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.548+0000 7f7e50e4f700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7e4c06a640 0x7f7e4c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:02.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.548+0000 7f7e50e4f700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e4c15c890 con 0x7f7e4c160790 2023-07-08T12:38:02.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.548+0000 7f7e50e4f700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e4c06f620 con 0x7f7e4c159c30 2023-07-08T12:38:02.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.548+0000 7f7e50e4f700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e4c15c730 con 0x7f7e4c06a640 2023-07-08T12:38:02.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.548+0000 7f7e49d9b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f7e4c160790 0x7f7e4c069f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36260/0 (socket says 172.21.15.12:36260) 2023-07-08T12:38:02.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.548+0000 7f7e49d9b700 1 -- 172.21.15.12:0/1949815283 learned_addr learned my addr 172.21.15.12:0/1949815283 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:02.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.548+0000 7f7e4ad9d700 1 --2- 172.21.15.12:0/1949815283 >> v2:172.21.15.160:3300/0 conn(0x7f7e4c06a640 0x7f7e4c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:02.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.548+0000 7f7e4959a700 1 -- 172.21.15.12:0/1949815283 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001575265 0 0) 0x7f7e4c15c890 con 0x7f7e4c160790 2023-07-08T12:38:02.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.548+0000 7f7e4959a700 1 -- 172.21.15.12:0/1949815283 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e340035b0 con 0x7f7e4c160790 2023-07-08T12:38:02.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.548+0000 7f7e4959a700 1 -- 172.21.15.12:0/1949815283 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389878769 0 0) 0x7f7e4c06f620 con 0x7f7e4c159c30 2023-07-08T12:38:02.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.548+0000 7f7e4959a700 1 -- 172.21.15.12:0/1949815283 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e4c15c890 con 0x7f7e4c159c30 2023-07-08T12:38:02.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.549+0000 7f7e4ad9d700 1 -- 172.21.15.12:0/1949815283 >> v1:172.21.15.160:6789/0 conn(0x7f7e4c159c30 legacy=0x7f7e4c06ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.549+0000 7f7e4ad9d700 1 -- 172.21.15.12:0/1949815283 >> v1:172.21.15.12:6789/0 conn(0x7f7e4c160790 legacy=0x7f7e4c069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.549+0000 7f7e4ad9d700 1 -- 172.21.15.12:0/1949815283 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4c160c90 con 0x7f7e4c06a640 2023-07-08T12:38:02.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.549+0000 7f7e4ad9d700 1 --2- 172.21.15.12:0/1949815283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e4c06a640 0x7f7e4c0cf050 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009d90 tx=0x7f7e4400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2996cff43202a67d server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:02.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.549+0000 7f7e4959a700 1 -- 172.21.15.12:0/1949815283 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4400e070 con 0x7f7e4c06a640 2023-07-08T12:38:02.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.549+0000 7f7e4959a700 1 -- 172.21.15.12:0/1949815283 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4400bb20 con 0x7f7e4c06a640 2023-07-08T12:38:02.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.549+0000 7f7e4959a700 1 -- 172.21.15.12:0/1949815283 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e44012430 con 0x7f7e4c06a640 2023-07-08T12:38:02.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.549+0000 7f7e50e4f700 1 -- 172.21.15.12:0/1949815283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e4c06a640 msgr2=0x7f7e4c0cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.549+0000 7f7e50e4f700 1 --2- 172.21.15.12:0/1949815283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e4c06a640 0x7f7e4c0cf050 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009d90 tx=0x7f7e4400b330 comp rx=0 tx=0).stop 2023-07-08T12:38:02.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.550+0000 7f7e50e4f700 1 -- 172.21.15.12:0/1949815283 shutdown_connections 2023-07-08T12:38:02.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.550+0000 7f7e50e4f700 1 --2- 172.21.15.12:0/1949815283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e4c06a640 0x7f7e4c0cf050 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.550+0000 7f7e50e4f700 1 -- 172.21.15.12:0/1949815283 >> 172.21.15.12:0/1949815283 conn(0x7f7e4c0cc290 msgr2=0x7f7e4c0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:02.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.550+0000 7f7e50e4f700 1 -- 172.21.15.12:0/1949815283 shutdown_connections 2023-07-08T12:38:02.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.550+0000 7f7e50e4f700 1 -- 172.21.15.12:0/1949815283 wait complete. 2023-07-08T12:38:02.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.551+0000 7f7e50e4f700 1 Processor -- start 2023-07-08T12:38:02.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.551+0000 7f7e50e4f700 1 -- start start 2023-07-08T12:38:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e50e4f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e4c06a640 0x7f7e4c0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e50e4f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e4c159c30 0x7f7e4c0c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e50e4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e4c160790 0x7f7e4c14dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e50e4f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c0c5cd0 con 0x7f7e4c160790 2023-07-08T12:38:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e50e4f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c0c43b0 con 0x7f7e4c06a640 2023-07-08T12:38:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e50e4f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c0c60f0 con 0x7f7e4c159c30 2023-07-08T12:38:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e4ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e4c160790 0x7f7e4c14dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e4ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e4c160790 0x7f7e4c14dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48874/0 (socket says 172.21.15.12:48874) 2023-07-08T12:38:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e4ad9d700 1 -- 172.21.15.12:0/3384351933 learned_addr learned my addr 172.21.15.12:0/3384351933 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e49d9b700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e4c159c30 0x7f7e4c0c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e4a59c700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e4c06a640 0x7f7e4c0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e4ad9d700 1 -- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e4c159c30 msgr2=0x7f7e4c0c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e4ad9d700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e4c159c30 0x7f7e4c0c3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e4ad9d700 1 -- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e4c06a640 msgr2=0x7f7e4c0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e4ad9d700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e4c06a640 0x7f7e4c0c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e4ad9d700 1 -- 172.21.15.12:0/3384351933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e44009a70 con 0x7f7e4c160790 2023-07-08T12:38:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.552+0000 7f7e4ad9d700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e4c160790 0x7f7e4c14dd90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c070c60 tx=0x7f7e44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.553+0000 7f7e3affd700 1 -- 172.21.15.12:0/3384351933 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4401c810 con 0x7f7e4c160790 2023-07-08T12:38:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.553+0000 7f7e3affd700 1 -- 172.21.15.12:0/3384351933 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e44020070 con 0x7f7e4c160790 2023-07-08T12:38:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.553+0000 7f7e3affd700 1 -- 172.21.15.12:0/3384351933 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4401cbb0 con 0x7f7e4c160790 2023-07-08T12:38:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.553+0000 7f7e50e4f700 1 -- 172.21.15.12:0/3384351933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e4c0c5050 con 0x7f7e4c160790 2023-07-08T12:38:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.553+0000 7f7e50e4f700 1 -- 172.21.15.12:0/3384351933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4c0c5610 con 0x7f7e4c160790 2023-07-08T12:38:02.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.554+0000 7f7e50e4f700 1 -- 172.21.15.12:0/3384351933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4c0c5e10 con 0x7f7e4c160790 2023-07-08T12:38:02.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.554+0000 7f7e3affd700 1 -- 172.21.15.12:0/3384351933 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e4401cd10 con 0x7f7e4c160790 2023-07-08T12:38:02.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.555+0000 7f7e3affd700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7e2c06ace0 0x7f7e2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:02.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.555+0000 7f7e3affd700 1 -- 172.21.15.12:0/3384351933 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7e44022070 con 0x7f7e4c160790 2023-07-08T12:38:02.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.555+0000 7f7e4a59c700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7e2c06ace0 0x7f7e2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:02.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.555+0000 7f7e4a59c700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7e2c06ace0 0x7f7e2c06d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e40007fd0 tx=0x7f7e40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:02.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.559+0000 7f7e3affd700 1 -- 172.21.15.12:0/3384351933 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7e44056800 con 0x7f7e4c160790 2023-07-08T12:38:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.759+0000 7f7e50e4f700 1 -- 172.21.15.12:0/3384351933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e4c04ea60 con 0x7f7e4c160790 2023-07-08T12:38:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.760+0000 7f7e3affd700 1 -- 172.21.15.12:0/3384351933 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14637 (secure 0 0 0) 0x7f7e44059e20 con 0x7f7e4c160790 2023-07-08T12:38:02.763 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:02.763 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:02.362751+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:02.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.762+0000 7f7e50e4f700 1 -- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7e2c06ace0 msgr2=0x7f7e2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.762+0000 7f7e50e4f700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7e2c06ace0 0x7f7e2c06d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e40007fd0 tx=0x7f7e40009040 comp rx=0 tx=0).stop 2023-07-08T12:38:02.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.762+0000 7f7e50e4f700 1 -- 172.21.15.12:0/3384351933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e4c160790 msgr2=0x7f7e4c14dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.762+0000 7f7e50e4f700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e4c160790 0x7f7e4c14dd90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c070c60 tx=0x7f7e44004ad0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.762+0000 7f7e50e4f700 1 -- 172.21.15.12:0/3384351933 shutdown_connections 2023-07-08T12:38:02.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.762+0000 7f7e50e4f700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7e2c06ace0 0x7f7e2c06d190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.762+0000 7f7e50e4f700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e4c160790 0x7f7e4c14dd90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.762+0000 7f7e50e4f700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e4c06a640 0x7f7e4c0c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.762+0000 7f7e50e4f700 1 --2- 172.21.15.12:0/3384351933 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e4c159c30 0x7f7e4c0c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.762+0000 7f7e50e4f700 1 -- 172.21.15.12:0/3384351933 >> 172.21.15.12:0/3384351933 conn(0x7f7e4c0cc290 msgr2=0x7f7e4c0cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:02.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.762+0000 7f7e50e4f700 1 -- 172.21.15.12:0/3384351933 shutdown_connections 2023-07-08T12:38:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.762+0000 7f7e50e4f700 1 -- 172.21.15.12:0/3384351933 wait complete. 2023-07-08T12:38:02.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-08T12:38:02.777 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:38:02.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.905+0000 7f74aab0b700 1 Processor -- start 2023-07-08T12:38:02.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.905+0000 7f74aab0b700 1 -- start start 2023-07-08T12:38:02.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74aab0b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74a4156a10 0x7f74a406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:02.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74aab0b700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a4160410 con 0x7f74a40cc240 2023-07-08T12:38:02.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74aab0b700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a4156de0 con 0x7f74a415d390 2023-07-08T12:38:02.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74aab0b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a406ec70 con 0x7f74a4156a10 2023-07-08T12:38:02.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a90a8700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f74a40cc240 0x7f74a40ae440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36278/0 (socket says 172.21.15.12:36278) 2023-07-08T12:38:02.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a90a8700 1 -- 172.21.15.12:0/3027385493 learned_addr learned my addr 172.21.15.12:0/3027385493 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:02.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a88a7700 1 --2- 172.21.15.12:0/3027385493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74a4156a10 0x7f74a406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:02.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183261068 0 0) 0x7f74a4160410 con 0x7f74a40cc240 2023-07-08T12:38:02.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74940035b0 con 0x7f74a40cc240 2023-07-08T12:38:02.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2957709935 0 0) 0x7f74940035b0 con 0x7f74a40cc240 2023-07-08T12:38:02.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74a4156a10 msgr2=0x7f74a406e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:02.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 --2- 172.21.15.12:0/3027385493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74a4156a10 0x7f74a406e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 >> v1:172.21.15.160:6789/0 conn(0x7f74a415d390 legacy=0x7f74a41608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a406e6b0 con 0x7f74a40cc240 2023-07-08T12:38:02.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f749c0019f0 con 0x7f74a40cc240 2023-07-08T12:38:02.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 >> v1:172.21.15.12:6789/0 conn(0x7f74a40cc240 legacy=0x7f74a40ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 --2- 172.21.15.12:0/3027385493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7494003a30 0x7f7494003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:02.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 --2- 172.21.15.12:0/3027385493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74a4156a10 0x7f7494004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:02.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 --2- 172.21.15.12:0/3027385493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7494004d90 0x7f7494005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:02.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a406ec70 con 0x7f7494004d90 2023-07-08T12:38:02.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749400bac0 con 0x7f7494003a30 2023-07-08T12:38:02.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749400bc50 con 0x7f74a4156a10 2023-07-08T12:38:02.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a90a8700 1 --2- 172.21.15.12:0/3027385493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7494004d90 0x7f7494005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:02.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a88a7700 1 --2- 172.21.15.12:0/3027385493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7494003a30 0x7f7494003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:02.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.906+0000 7f74a3fff700 1 --2- 172.21.15.12:0/3027385493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74a4156a10 0x7f7494004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:02.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74a90a8700 1 -- 172.21.15.12:0/3027385493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74a4156a10 msgr2=0x7f7494004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:02.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74a90a8700 1 --2- 172.21.15.12:0/3027385493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74a4156a10 0x7f7494004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74a90a8700 1 -- 172.21.15.12:0/3027385493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7494003a30 msgr2=0x7f7494003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:02.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74a90a8700 1 --2- 172.21.15.12:0/3027385493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7494003a30 0x7f7494003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74a90a8700 1 -- 172.21.15.12:0/3027385493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940035b0 con 0x7f7494004d90 2023-07-08T12:38:02.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74a90a8700 1 --2- 172.21.15.12:0/3027385493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7494004d90 0x7f7494005200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f749c00a830 tx=0x7f7494005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2580399d7624e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:02.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c00a2f0 con 0x7f7494004d90 2023-07-08T12:38:02.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c012620 con 0x7f7494004d90 2023-07-08T12:38:02.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74a37fe700 1 -- 172.21.15.12:0/3027385493 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c0129d0 con 0x7f7494004d90 2023-07-08T12:38:02.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74aab0b700 1 -- 172.21.15.12:0/3027385493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7494004d90 msgr2=0x7f7494005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74aab0b700 1 --2- 172.21.15.12:0/3027385493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7494004d90 0x7f7494005200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f749c00a830 tx=0x7f7494005880 comp rx=0 tx=0).stop 2023-07-08T12:38:02.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74a90a8700 1 -- 172.21.15.12:0/3027385493 reap_dead start 2023-07-08T12:38:02.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74aab0b700 1 -- 172.21.15.12:0/3027385493 shutdown_connections 2023-07-08T12:38:02.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.907+0000 7f74aab0b700 1 -- 172.21.15.12:0/3027385493 >> 172.21.15.12:0/3027385493 conn(0x7f74a4098b50 msgr2=0x7f74a406b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:02.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.908+0000 7f74aab0b700 1 -- 172.21.15.12:0/3027385493 shutdown_connections 2023-07-08T12:38:02.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.908+0000 7f74aab0b700 1 -- 172.21.15.12:0/3027385493 wait complete. 2023-07-08T12:38:02.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.909+0000 7f74aab0b700 1 Processor -- start 2023-07-08T12:38:02.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.909+0000 7f74aab0b700 1 -- start start 2023-07-08T12:38:02.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74aab0b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7494004d90 0x7f74a40af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:02.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74aab0b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74a40af6d0 0x7f74a40b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:02.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74aab0b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74a40b2ba0 0x7f74a4142e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:02.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74aab0b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a40b3020 con 0x7f7494004d90 2023-07-08T12:38:02.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74aab0b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a40b0910 con 0x7f74a40b2ba0 2023-07-08T12:38:02.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74aab0b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a40afd00 con 0x7f74a40af6d0 2023-07-08T12:38:02.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a88a7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7494004d90 0x7f74a40af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:02.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a88a7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7494004d90 0x7f74a40af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48890/0 (socket says 172.21.15.12:48890) 2023-07-08T12:38:02.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a88a7700 1 -- 172.21.15.12:0/576298712 learned_addr learned my addr 172.21.15.12:0/576298712 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:02.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a90a8700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74a40b2ba0 0x7f74a4142e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:02.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a3fff700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74a40af6d0 0x7f74a40b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:02.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a88a7700 1 -- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74a40af6d0 msgr2=0x7f74a40b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a88a7700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74a40af6d0 0x7f74a40b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a88a7700 1 -- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74a40b2ba0 msgr2=0x7f74a4142e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:02.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a88a7700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74a40b2ba0 0x7f74a4142e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:02.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a88a7700 1 -- 172.21.15.12:0/576298712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a40affe0 con 0x7f7494004d90 2023-07-08T12:38:02.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a90a8700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74a40b2ba0 0x7f74a4142e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:02.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a88a7700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7494004d90 0x7f74a40af190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f74980119f0 tx=0x7f7498011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:02.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a17fa700 1 -- 172.21.15.12:0/576298712 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7498005c30 con 0x7f7494004d90 2023-07-08T12:38:02.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a17fa700 1 -- 172.21.15.12:0/576298712 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7498005d90 con 0x7f7494004d90 2023-07-08T12:38:02.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74aab0b700 1 -- 172.21.15.12:0/576298712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a40b02c0 con 0x7f7494004d90 2023-07-08T12:38:02.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74a17fa700 1 -- 172.21.15.12:0/576298712 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749800f8f0 con 0x7f7494004d90 2023-07-08T12:38:02.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.910+0000 7f74aab0b700 1 -- 172.21.15.12:0/576298712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a4156110 con 0x7f7494004d90 2023-07-08T12:38:02.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.911+0000 7f74aab0b700 1 -- 172.21.15.12:0/576298712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a40afaa0 con 0x7f7494004d90 2023-07-08T12:38:02.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.912+0000 7f74a17fa700 1 -- 172.21.15.12:0/576298712 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f749800fa50 con 0x7f7494004d90 2023-07-08T12:38:02.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.912+0000 7f74a17fa700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f748806ace0 0x7f748806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:02.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.912+0000 7f74a17fa700 1 -- 172.21.15.12:0/576298712 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f74980899b0 con 0x7f7494004d90 2023-07-08T12:38:02.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.912+0000 7f74a3fff700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f748806ace0 0x7f748806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:02.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.913+0000 7f74a3fff700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f748806ace0 0x7f748806d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7490007fd0 tx=0x7f7490009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:02.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:02.917+0000 7f74a17fa700 1 -- 172.21.15.12:0/576298712 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7498054db0 con 0x7f7494004d90 2023-07-08T12:38:03.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.114+0000 7f74aab0b700 1 -- 172.21.15.12:0/576298712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74a4066af0 con 0x7f7494004d90 2023-07-08T12:38:03.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.115+0000 7f74a17fa700 1 -- 172.21.15.12:0/576298712 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74980583d0 con 0x7f7494004d90 2023-07-08T12:38:03.117 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:38:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.117+0000 7f74aab0b700 1 -- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f748806ace0 msgr2=0x7f748806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.117+0000 7f74aab0b700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f748806ace0 0x7f748806d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7490007fd0 tx=0x7f7490009040 comp rx=0 tx=0).stop 2023-07-08T12:38:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.117+0000 7f74aab0b700 1 -- 172.21.15.12:0/576298712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7494004d90 msgr2=0x7f74a40af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.117+0000 7f74aab0b700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7494004d90 0x7f74a40af190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f74980119f0 tx=0x7f7498011db0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.117+0000 7f74aab0b700 1 -- 172.21.15.12:0/576298712 shutdown_connections 2023-07-08T12:38:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.117+0000 7f74aab0b700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f748806ace0 0x7f748806d190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.117+0000 7f74aab0b700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74a40b2ba0 0x7f74a4142e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.117+0000 7f74aab0b700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7494004d90 0x7f74a40af190 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.117+0000 7f74aab0b700 1 --2- 172.21.15.12:0/576298712 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74a40af6d0 0x7f74a40b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.117+0000 7f74aab0b700 1 -- 172.21.15.12:0/576298712 >> 172.21.15.12:0/576298712 conn(0x7f74a4098b50 msgr2=0x7f74a415a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:03.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.117+0000 7f74aab0b700 1 -- 172.21.15.12:0/576298712 shutdown_connections 2023-07-08T12:38:03.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.117+0000 7f74aab0b700 1 -- 172.21.15.12:0/576298712 wait complete. 2023-07-08T12:38:03.131 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:38:03.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.258+0000 7f2915a27700 1 Processor -- start 2023-07-08T12:38:03.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.258+0000 7f2915a27700 1 -- start start 2023-07-08T12:38:03.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f2915a27700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f29100b8430 0x7f29100cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:03.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f2915a27700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f29100d3730 0x7f29100cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f2915a27700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29100d0b20 con 0x7f29100cfa90 2023-07-08T12:38:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f2915a27700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29100b8800 con 0x7f29100b8430 2023-07-08T12:38:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f2915a27700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29100cfe90 con 0x7f29100d3730 2023-07-08T12:38:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f290effd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f29100b8430 0x7f29100cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f290effd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f29100b8430 0x7f29100cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48898/0 (socket says 172.21.15.12:48898) 2023-07-08T12:38:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f290effd700 1 -- 172.21.15.12:0/309383388 learned_addr learned my addr 172.21.15.12:0/309383388 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f290e7fc700 1 --2- 172.21.15.12:0/309383388 >> v2:172.21.15.160:3300/0 conn(0x7f29100d3730 0x7f29100cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f290effd700 1 -- 172.21.15.12:0/309383388 >> v2:172.21.15.160:3300/0 conn(0x7f29100d3730 msgr2=0x7f29100cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:03.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f290effd700 1 --2- 172.21.15.12:0/309383388 >> v2:172.21.15.160:3300/0 conn(0x7f29100d3730 0x7f29100cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f290effd700 1 -- 172.21.15.12:0/309383388 >> v1:172.21.15.160:6789/0 conn(0x7f29100cfa90 legacy=0x7f291015feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:03.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f290effd700 1 -- 172.21.15.12:0/309383388 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29100b4ee0 con 0x7f29100b8430 2023-07-08T12:38:03.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.259+0000 7f290effd700 1 --2- 172.21.15.12:0/309383388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29100b8430 0x7f29100cd320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2910160f50 tx=0x7f2904009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88807a0ab398fe18 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:03.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.260+0000 7f290dffb700 1 -- 172.21.15.12:0/309383388 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904011830 con 0x7f29100b8430 2023-07-08T12:38:03.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.260+0000 7f290dffb700 1 -- 172.21.15.12:0/309383388 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2904011990 con 0x7f29100b8430 2023-07-08T12:38:03.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.260+0000 7f290dffb700 1 -- 172.21.15.12:0/309383388 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904011d20 con 0x7f29100b8430 2023-07-08T12:38:03.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.260+0000 7f2915a27700 1 -- 172.21.15.12:0/309383388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29100b8430 msgr2=0x7f29100cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:03.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.260+0000 7f2915a27700 1 --2- 172.21.15.12:0/309383388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29100b8430 0x7f29100cd320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2910160f50 tx=0x7f2904009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.260+0000 7f2915a27700 1 -- 172.21.15.12:0/309383388 shutdown_connections 2023-07-08T12:38:03.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.260+0000 7f2915a27700 1 --2- 172.21.15.12:0/309383388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29100b8430 0x7f29100cd320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.260+0000 7f2915a27700 1 --2- 172.21.15.12:0/309383388 >> v2:172.21.15.160:3300/0 conn(0x7f29100d3730 0x7f29100cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.260+0000 7f2915a27700 1 -- 172.21.15.12:0/309383388 >> 172.21.15.12:0/309383388 conn(0x7f291006b3f0 msgr2=0x7f291006b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:03.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.260+0000 7f2915a27700 1 -- 172.21.15.12:0/309383388 shutdown_connections 2023-07-08T12:38:03.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.260+0000 7f2915a27700 1 -- 172.21.15.12:0/309383388 wait complete. 2023-07-08T12:38:03.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.261+0000 7f2915a27700 1 Processor -- start 2023-07-08T12:38:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.262+0000 7f2915a27700 1 -- start start 2023-07-08T12:38:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.262+0000 7f2915a27700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29100b8430 0x7f29100cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.262+0000 7f2915a27700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29100cfa90 0x7f29100c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.262+0000 7f2915a27700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29100d3730 0x7f29100c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.262+0000 7f2915a27700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291015b020 con 0x7f29100cfa90 2023-07-08T12:38:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.262+0000 7f2915a27700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29100c7bf0 con 0x7f29100b8430 2023-07-08T12:38:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.262+0000 7f2915a27700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29100c7ed0 con 0x7f29100d3730 2023-07-08T12:38:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.262+0000 7f290e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29100cfa90 0x7f29100c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.262+0000 7f290e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29100cfa90 0x7f29100c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48906/0 (socket says 172.21.15.12:48906) 2023-07-08T12:38:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.262+0000 7f290e7fc700 1 -- 172.21.15.12:0/4280056175 learned_addr learned my addr 172.21.15.12:0/4280056175 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.262+0000 7f290f7fe700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29100d3730 0x7f29100c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.262+0000 7f290effd700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29100b8430 0x7f29100cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.263+0000 7f290e7fc700 1 -- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29100d3730 msgr2=0x7f29100c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.263+0000 7f290e7fc700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29100d3730 0x7f29100c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.263+0000 7f290e7fc700 1 -- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29100b8430 msgr2=0x7f29100cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.263+0000 7f290e7fc700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29100b8430 0x7f29100cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.263+0000 7f290e7fc700 1 -- 172.21.15.12:0/4280056175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2904009550 con 0x7f29100cfa90 2023-07-08T12:38:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.263+0000 7f290e7fc700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29100cfa90 0x7f29100c7080 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f290000e9b0 tx=0x7f290000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.263+0000 7f28f7fff700 1 -- 172.21.15.12:0/4280056175 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900009710 con 0x7f29100cfa90 2023-07-08T12:38:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.263+0000 7f28f7fff700 1 -- 172.21.15.12:0/4280056175 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290000ee50 con 0x7f29100cfa90 2023-07-08T12:38:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.263+0000 7f2915a27700 1 -- 172.21.15.12:0/4280056175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291014abc0 con 0x7f29100cfa90 2023-07-08T12:38:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.263+0000 7f28f7fff700 1 -- 172.21.15.12:0/4280056175 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290000ca50 con 0x7f29100cfa90 2023-07-08T12:38:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.263+0000 7f2915a27700 1 -- 172.21.15.12:0/4280056175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291014b180 con 0x7f29100cfa90 2023-07-08T12:38:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.264+0000 7f2915a27700 1 -- 172.21.15.12:0/4280056175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291014b370 con 0x7f29100cfa90 2023-07-08T12:38:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.264+0000 7f28f7fff700 1 -- 172.21.15.12:0/4280056175 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f290000cbb0 con 0x7f29100cfa90 2023-07-08T12:38:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.265+0000 7f28f7fff700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f28fc06ad10 0x7f28fc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.265+0000 7f28f7fff700 1 -- 172.21.15.12:0/4280056175 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2900089080 con 0x7f29100cfa90 2023-07-08T12:38:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.265+0000 7f290effd700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f28fc06ad10 0x7f28fc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.265+0000 7f290effd700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f28fc06ad10 0x7f28fc06d1c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2904014040 tx=0x7f2904008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:03.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.269+0000 7f28f7fff700 1 -- 172.21.15.12:0/4280056175 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2900057fa0 con 0x7f29100cfa90 2023-07-08T12:38:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.473+0000 7f2915a27700 1 -- 172.21.15.12:0/4280056175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f291004ea60 con 0x7f29100cfa90 2023-07-08T12:38:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.473+0000 7f28f7fff700 1 -- 172.21.15.12:0/4280056175 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f290000ce60 con 0x7f29100cfa90 2023-07-08T12:38:03.476 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:38:03.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.475+0000 7f2915a27700 1 -- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f28fc06ad10 msgr2=0x7f28fc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:03.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.475+0000 7f2915a27700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f28fc06ad10 0x7f28fc06d1c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2904014040 tx=0x7f2904008030 comp rx=0 tx=0).stop 2023-07-08T12:38:03.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.475+0000 7f2915a27700 1 -- 172.21.15.12:0/4280056175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29100cfa90 msgr2=0x7f29100c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:03.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.475+0000 7f2915a27700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29100cfa90 0x7f29100c7080 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f290000e9b0 tx=0x7f290000ecc0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.475+0000 7f2915a27700 1 -- 172.21.15.12:0/4280056175 shutdown_connections 2023-07-08T12:38:03.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.475+0000 7f2915a27700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f28fc06ad10 0x7f28fc06d1c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.475+0000 7f2915a27700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29100d3730 0x7f29100c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.475+0000 7f2915a27700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29100b8430 0x7f29100cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.475+0000 7f2915a27700 1 --2- 172.21.15.12:0/4280056175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29100cfa90 0x7f29100c7080 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.475+0000 7f2915a27700 1 -- 172.21.15.12:0/4280056175 >> 172.21.15.12:0/4280056175 conn(0x7f291006b3f0 msgr2=0x7f29100b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:03.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.475+0000 7f2915a27700 1 -- 172.21.15.12:0/4280056175 shutdown_connections 2023-07-08T12:38:03.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.475+0000 7f2915a27700 1 -- 172.21.15.12:0/4280056175 wait complete. 2023-07-08T12:38:03.490 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-08T12:38:03.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.613+0000 7ff1cb93a700 1 Processor -- start 2023-07-08T12:38:03.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.614+0000 7ff1cb93a700 1 -- start start 2023-07-08T12:38:03.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.614+0000 7ff1cb93a700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff1c40b1c70 0x7ff1c40ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:03.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.614+0000 7ff1cb93a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff1c415d6c0 0x7ff1c4071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:03.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.614+0000 7ff1cb93a700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c415d230 con 0x7ff1c40ce1b0 2023-07-08T12:38:03.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.614+0000 7ff1cb93a700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c40c9e60 con 0x7ff1c415d6c0 2023-07-08T12:38:03.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.614+0000 7ff1cb93a700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c40ceac0 con 0x7ff1c40b1c70 2023-07-08T12:38:03.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.614+0000 7ff1c9ed7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff1c415d6c0 0x7ff1c4071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:03.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.614+0000 7ff1c9ed7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff1c415d6c0 0x7ff1c4071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48918/0 (socket says 172.21.15.12:48918) 2023-07-08T12:38:03.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.614+0000 7ff1c9ed7700 1 -- 172.21.15.12:0/2887302264 learned_addr learned my addr 172.21.15.12:0/2887302264 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:03.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.615+0000 7ff1c8ed5700 1 --2- 172.21.15.12:0/2887302264 >> v2:172.21.15.160:3300/0 conn(0x7ff1c40b1c70 0x7ff1c40ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:03.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.615+0000 7ff1c9ed7700 1 -- 172.21.15.12:0/2887302264 >> v2:172.21.15.160:3300/0 conn(0x7ff1c40b1c70 msgr2=0x7ff1c40ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:03.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.615+0000 7ff1c9ed7700 1 --2- 172.21.15.12:0/2887302264 >> v2:172.21.15.160:3300/0 conn(0x7ff1c40b1c70 0x7ff1c40ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.615+0000 7ff1c9ed7700 1 -- 172.21.15.12:0/2887302264 >> v1:172.21.15.160:6789/0 conn(0x7ff1c40ce1b0 legacy=0x7ff1c40cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:03.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.615+0000 7ff1c9ed7700 1 -- 172.21.15.12:0/2887302264 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c40cdbc0 con 0x7ff1c415d6c0 2023-07-08T12:38:03.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.615+0000 7ff1c8ed5700 1 --2- 172.21.15.12:0/2887302264 >> v2:172.21.15.160:3300/0 conn(0x7ff1c40b1c70 0x7ff1c40ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:03.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.615+0000 7ff1c9ed7700 1 --2- 172.21.15.12:0/2887302264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1c415d6c0 0x7ff1c4071530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff1c406ff80 tx=0x7ff1bc00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f253178c91302a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:03.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.615+0000 7ff1b7fff700 1 -- 172.21.15.12:0/2887302264 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc010070 con 0x7ff1c415d6c0 2023-07-08T12:38:03.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.615+0000 7ff1b7fff700 1 -- 172.21.15.12:0/2887302264 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc00dd50 con 0x7ff1c415d6c0 2023-07-08T12:38:03.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.615+0000 7ff1b7fff700 1 -- 172.21.15.12:0/2887302264 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc00b310 con 0x7ff1c415d6c0 2023-07-08T12:38:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.616+0000 7ff1cb93a700 1 -- 172.21.15.12:0/2887302264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1c415d6c0 msgr2=0x7ff1c4071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.616+0000 7ff1cb93a700 1 --2- 172.21.15.12:0/2887302264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1c415d6c0 0x7ff1c4071530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff1c406ff80 tx=0x7ff1bc00d430 comp rx=0 tx=0).stop 2023-07-08T12:38:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.616+0000 7ff1cb93a700 1 -- 172.21.15.12:0/2887302264 shutdown_connections 2023-07-08T12:38:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.616+0000 7ff1cb93a700 1 --2- 172.21.15.12:0/2887302264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1c415d6c0 0x7ff1c4071530 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.616+0000 7ff1cb93a700 1 --2- 172.21.15.12:0/2887302264 >> v2:172.21.15.160:3300/0 conn(0x7ff1c40b1c70 0x7ff1c40ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.616+0000 7ff1cb93a700 1 -- 172.21.15.12:0/2887302264 >> 172.21.15.12:0/2887302264 conn(0x7ff1c40cc290 msgr2=0x7ff1c40cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.616+0000 7ff1cb93a700 1 -- 172.21.15.12:0/2887302264 shutdown_connections 2023-07-08T12:38:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.616+0000 7ff1cb93a700 1 -- 172.21.15.12:0/2887302264 wait complete. 2023-07-08T12:38:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.617+0000 7ff1cb93a700 1 Processor -- start 2023-07-08T12:38:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.617+0000 7ff1cb93a700 1 -- start start 2023-07-08T12:38:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.618+0000 7ff1cb93a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1c40b1c70 0x7ff1c4152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.618+0000 7ff1cb93a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1c40ce1b0 0x7ff1c40b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.618+0000 7ff1cb93a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1c415d6c0 0x7ff1c40b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:03.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.618+0000 7ff1cb93a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40ab8b0 con 0x7ff1c415d6c0 2023-07-08T12:38:03.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.618+0000 7ff1cb93a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c41534d0 con 0x7ff1c40ce1b0 2023-07-08T12:38:03.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.618+0000 7ff1cb93a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40adce0 con 0x7ff1c40b1c70 2023-07-08T12:38:03.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1c9ed7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1c415d6c0 0x7ff1c40b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:03.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1c9ed7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1c415d6c0 0x7ff1c40b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48924/0 (socket says 172.21.15.12:48924) 2023-07-08T12:38:03.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1c9ed7700 1 -- 172.21.15.12:0/4097647197 learned_addr learned my addr 172.21.15.12:0/4097647197 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:03.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1c96d6700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1c40b1c70 0x7ff1c4152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1c8ed5700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1c40ce1b0 0x7ff1c40b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1c9ed7700 1 -- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1c40b1c70 msgr2=0x7ff1c4152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1c9ed7700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1c40b1c70 0x7ff1c4152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1c9ed7700 1 -- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1c40ce1b0 msgr2=0x7ff1c40b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1c9ed7700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1c40ce1b0 0x7ff1c40b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1c9ed7700 1 -- 172.21.15.12:0/4097647197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc00bd40 con 0x7ff1c415d6c0 2023-07-08T12:38:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1c8ed5700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1c40ce1b0 0x7ff1c40b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1c96d6700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1c40b1c70 0x7ff1c4152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1c9ed7700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1c415d6c0 0x7ff1c40b41f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4070c60 tx=0x7ff1bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1b5ffb700 1 -- 172.21.15.12:0/4097647197 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc009a10 con 0x7ff1c415d6c0 2023-07-08T12:38:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1b5ffb700 1 -- 172.21.15.12:0/4097647197 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc009b90 con 0x7ff1c415d6c0 2023-07-08T12:38:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1cb93a700 1 -- 172.21.15.12:0/4097647197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c40adfc0 con 0x7ff1c415d6c0 2023-07-08T12:38:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1b5ffb700 1 -- 172.21.15.12:0/4097647197 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc004030 con 0x7ff1c415d6c0 2023-07-08T12:38:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.619+0000 7ff1cb93a700 1 -- 172.21.15.12:0/4097647197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c40abb00 con 0x7ff1c415d6c0 2023-07-08T12:38:03.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.620+0000 7ff1cb93a700 1 -- 172.21.15.12:0/4097647197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c404ea60 con 0x7ff1c415d6c0 2023-07-08T12:38:03.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.621+0000 7ff1b5ffb700 1 -- 172.21.15.12:0/4097647197 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff1bc009050 con 0x7ff1c415d6c0 2023-07-08T12:38:03.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.621+0000 7ff1b5ffb700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff1a806ace0 0x7ff1a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:03.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.621+0000 7ff1b5ffb700 1 -- 172.21.15.12:0/4097647197 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff1bc08a7d0 con 0x7ff1c415d6c0 2023-07-08T12:38:03.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.622+0000 7ff1c96d6700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff1a806ace0 0x7ff1a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:03.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.622+0000 7ff1c96d6700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff1a806ace0 0x7ff1a806d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0003f40 tx=0x7ff1c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:03.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.626+0000 7ff1b5ffb700 1 -- 172.21.15.12:0/4097647197 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff1bc055bc0 con 0x7ff1c415d6c0 2023-07-08T12:38:03.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.867+0000 7ff1cb93a700 1 -- 172.21.15.12:0/4097647197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1c40c6680 con 0x7ff1c415d6c0 2023-07-08T12:38:03.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.867+0000 7ff1b5ffb700 1 -- 172.21.15.12:0/4097647197 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7ff1bc019de0 con 0x7ff1c415d6c0 2023-07-08T12:38:03.870 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:03.870 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3238006310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3238006310},{"type":"v1","addr":"172.21.15.160:6836","nonce":3238006310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3864661454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3864661454},{"type":"v1","addr":"172.21.15.160:6837","nonce":3864661454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6834/946118830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":946118830},{"type":"v1","addr":"172.21.15.12:6834","nonce":946118830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/3301134992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3301134992},{"type":"v1","addr":"172.21.15.12:6835","nonce":3301134992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-08T12:38:03.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.869+0000 7ff1cb93a700 1 -- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff1a806ace0 msgr2=0x7ff1a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:03.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.869+0000 7ff1cb93a700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff1a806ace0 0x7ff1a806d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0003f40 tx=0x7ff1c0009040 comp rx=0 tx=0).stop 2023-07-08T12:38:03.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.869+0000 7ff1cb93a700 1 -- 172.21.15.12:0/4097647197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1c415d6c0 msgr2=0x7ff1c40b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:03.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.869+0000 7ff1cb93a700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1c415d6c0 0x7ff1c40b41f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4070c60 tx=0x7ff1bc004ad0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.869+0000 7ff1cb93a700 1 -- 172.21.15.12:0/4097647197 shutdown_connections 2023-07-08T12:38:03.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.869+0000 7ff1cb93a700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff1a806ace0 0x7ff1a806d190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.869+0000 7ff1cb93a700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1c415d6c0 0x7ff1c40b41f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.869+0000 7ff1cb93a700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1c40b1c70 0x7ff1c4152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.869+0000 7ff1cb93a700 1 --2- 172.21.15.12:0/4097647197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1c40ce1b0 0x7ff1c40b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:03.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.869+0000 7ff1cb93a700 1 -- 172.21.15.12:0/4097647197 >> 172.21.15.12:0/4097647197 conn(0x7ff1c40cc290 msgr2=0x7ff1c4071d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:03.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.869+0000 7ff1cb93a700 1 -- 172.21.15.12:0/4097647197 shutdown_connections 2023-07-08T12:38:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:03.869+0000 7ff1cb93a700 1 -- 172.21.15.12:0/4097647197 wait complete. 2023-07-08T12:38:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 3 2023-07-08T12:38:03.884 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.006+0000 7f6f32a36700 1 Processor -- start 2023-07-08T12:38:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.006+0000 7f6f32a36700 1 -- start start 2023-07-08T12:38:04.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.007+0000 7f6f32a36700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6f2c160790 0x7f6f2c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.007+0000 7f6f32a36700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2c15c890 con 0x7f6f2c159c30 2023-07-08T12:38:04.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.007+0000 7f6f32a36700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2c06f620 con 0x7f6f2c069f30 2023-07-08T12:38:04.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.007+0000 7f6f32a36700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2c15c730 con 0x7f6f2c160790 2023-07-08T12:38:04.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.007+0000 7f6f2bfff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6f2c159c30 0x7f6f2c06ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36324/0 (socket says 172.21.15.12:36324) 2023-07-08T12:38:04.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.007+0000 7f6f2bfff700 1 -- 172.21.15.12:0/2710380261 learned_addr learned my addr 172.21.15.12:0/2710380261 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:04.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2b7fe700 1 --2- 172.21.15.12:0/2710380261 >> v2:172.21.15.160:3300/0 conn(0x7f6f2c160790 0x7f6f2c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151893626 0 0) 0x7f6f2c15c890 con 0x7f6f2c159c30 2023-07-08T12:38:04.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f180035b0 con 0x7f6f2c159c30 2023-07-08T12:38:04.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1070242066 0 0) 0x7f6f180035b0 con 0x7f6f2c159c30 2023-07-08T12:38:04.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 >> v2:172.21.15.160:3300/0 conn(0x7f6f2c160790 msgr2=0x7f6f2c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:04.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 --2- 172.21.15.12:0/2710380261 >> v2:172.21.15.160:3300/0 conn(0x7f6f2c160790 0x7f6f2c0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 >> v1:172.21.15.160:6789/0 conn(0x7f6f2c069f30 legacy=0x7f6f2c15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c160c90 con 0x7f6f2c159c30 2023-07-08T12:38:04.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f6f1c001670 con 0x7f6f2c159c30 2023-07-08T12:38:04.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 >> v1:172.21.15.12:6789/0 conn(0x7f6f2c159c30 legacy=0x7f6f2c06ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 --2- 172.21.15.12:0/2710380261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f18003a00 0x7f6f18003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 --2- 172.21.15.12:0/2710380261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f18004850 0x7f6f18004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 --2- 172.21.15.12:0/2710380261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f18005200 0x7f6f1800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c15c730 con 0x7f6f18003a00 2023-07-08T12:38:04.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1800c080 con 0x7f6f18004850 2023-07-08T12:38:04.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1800c240 con 0x7f6f18005200 2023-07-08T12:38:04.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2bfff700 1 --2- 172.21.15.12:0/2710380261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f18003a00 0x7f6f18003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2b7fe700 1 --2- 172.21.15.12:0/2710380261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f18004850 0x7f6f18004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f30fd3700 1 --2- 172.21.15.12:0/2710380261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f18005200 0x7f6f1800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2bfff700 1 -- 172.21.15.12:0/2710380261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f18005200 msgr2=0x7f6f1800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:04.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2bfff700 1 --2- 172.21.15.12:0/2710380261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f18005200 0x7f6f1800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2bfff700 1 -- 172.21.15.12:0/2710380261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f18004850 msgr2=0x7f6f18004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:04.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2bfff700 1 --2- 172.21.15.12:0/2710380261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f18004850 0x7f6f18004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2bfff700 1 -- 172.21.15.12:0/2710380261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f180035b0 con 0x7f6f18003a00 2023-07-08T12:38:04.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f30fd3700 1 -- 172.21.15.12:0/2710380261 reap_dead start 2023-07-08T12:38:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.008+0000 7f6f2bfff700 1 --2- 172.21.15.12:0/2710380261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f18003a00 0x7f6f18003e10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00c240 tx=0x7f6f18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d48acfc6787f8a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.009+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c007c10 con 0x7f6f18003a00 2023-07-08T12:38:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.009+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c00cde0 con 0x7f6f18003a00 2023-07-08T12:38:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.009+0000 7f6f2affd700 1 -- 172.21.15.12:0/2710380261 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c005ba0 con 0x7f6f18003a00 2023-07-08T12:38:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.009+0000 7f6f32a36700 1 -- 172.21.15.12:0/2710380261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f18003a00 msgr2=0x7f6f18003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.009+0000 7f6f32a36700 1 --2- 172.21.15.12:0/2710380261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f18003a00 0x7f6f18003e10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00c240 tx=0x7f6f18005880 comp rx=0 tx=0).stop 2023-07-08T12:38:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.009+0000 7f6f32a36700 1 -- 172.21.15.12:0/2710380261 shutdown_connections 2023-07-08T12:38:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.009+0000 7f6f32a36700 1 --2- 172.21.15.12:0/2710380261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f18003a00 0x7f6f18003e10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.009+0000 7f6f32a36700 1 -- 172.21.15.12:0/2710380261 >> 172.21.15.12:0/2710380261 conn(0x7f6f2c0cc290 msgr2=0x7f6f2c0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.009+0000 7f6f32a36700 1 -- 172.21.15.12:0/2710380261 shutdown_connections 2023-07-08T12:38:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.009+0000 7f6f32a36700 1 -- 172.21.15.12:0/2710380261 wait complete. 2023-07-08T12:38:04.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.010+0000 7f6f32a36700 1 Processor -- start 2023-07-08T12:38:04.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.011+0000 7f6f32a36700 1 -- start start 2023-07-08T12:38:04.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.011+0000 7f6f32a36700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f18003a00 0x7f6f2c0af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.011+0000 7f6f32a36700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f2c0c5c50 0x7f6f2c0c2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.011+0000 7f6f32a36700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f2c0b08f0 0x7f6f2c0b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.011+0000 7f6f32a36700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0b04d0 con 0x7f6f2c0c5c50 2023-07-08T12:38:04.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.011+0000 7f6f32a36700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0c2bb0 con 0x7f6f2c0b08f0 2023-07-08T12:38:04.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.011+0000 7f6f32a36700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0af850 con 0x7f6f18003a00 2023-07-08T12:38:04.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.011+0000 7f6f2b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f2c0c5c50 0x7f6f2c0c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.011+0000 7f6f2b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f2c0c5c50 0x7f6f2c0c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48942/0 (socket says 172.21.15.12:48942) 2023-07-08T12:38:04.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.011+0000 7f6f2b7fe700 1 -- 172.21.15.12:0/1826072611 learned_addr learned my addr 172.21.15.12:0/1826072611 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:04.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.012+0000 7f6f2bfff700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f18003a00 0x7f6f2c0af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.012+0000 7f6f30fd3700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f2c0b08f0 0x7f6f2c0b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.012+0000 7f6f2b7fe700 1 -- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f18003a00 msgr2=0x7f6f2c0af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.012+0000 7f6f2b7fe700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f18003a00 0x7f6f2c0af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.012+0000 7f6f2b7fe700 1 -- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f2c0b08f0 msgr2=0x7f6f2c0b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.012+0000 7f6f2b7fe700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f2c0b08f0 0x7f6f2c0b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.012+0000 7f6f2b7fe700 1 -- 172.21.15.12:0/1826072611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c0afb30 con 0x7f6f2c0c5c50 2023-07-08T12:38:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.012+0000 7f6f2b7fe700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f2c0c5c50 0x7f6f2c0c2670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6f14011a30 tx=0x7f6f14011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.012+0000 7f6f28ff9700 1 -- 172.21.15.12:0/1826072611 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14011ed0 con 0x7f6f2c0c5c50 2023-07-08T12:38:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.012+0000 7f6f28ff9700 1 -- 172.21.15.12:0/1826072611 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f14004240 con 0x7f6f2c0c5c50 2023-07-08T12:38:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.012+0000 7f6f32a36700 1 -- 172.21.15.12:0/1826072611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2c0afe10 con 0x7f6f2c0c5c50 2023-07-08T12:38:04.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.012+0000 7f6f28ff9700 1 -- 172.21.15.12:0/1826072611 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400f730 con 0x7f6f2c0c5c50 2023-07-08T12:38:04.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.012+0000 7f6f32a36700 1 -- 172.21.15.12:0/1826072611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c0c60c0 con 0x7f6f2c0c5c50 2023-07-08T12:38:04.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.013+0000 7f6f32a36700 1 -- 172.21.15.12:0/1826072611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2c0b0610 con 0x7f6f2c0c5c50 2023-07-08T12:38:04.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.014+0000 7f6f28ff9700 1 -- 172.21.15.12:0/1826072611 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6f1400f890 con 0x7f6f2c0c5c50 2023-07-08T12:38:04.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.014+0000 7f6f28ff9700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f0c06ad10 0x7f6f0c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.014+0000 7f6f28ff9700 1 -- 172.21.15.12:0/1826072611 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6f140894a0 con 0x7f6f2c0c5c50 2023-07-08T12:38:04.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.014+0000 7f6f2bfff700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f0c06ad10 0x7f6f0c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.015+0000 7f6f2bfff700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f0c06ad10 0x7f6f0c06d1c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c000c00 tx=0x7f6f1c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:04.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.018+0000 7f6f28ff9700 1 -- 172.21.15.12:0/1826072611 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6f2c0b0610 con 0x7f6f2c0c5c50 2023-07-08T12:38:04.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.215+0000 7f6f32a36700 1 -- 172.21.15.12:0/1826072611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f2c0b0610 con 0x7f6f2c0c5c50 2023-07-08T12:38:04.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.216+0000 7f6f28ff9700 1 -- 172.21.15.12:0/1826072611 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+14624 (secure 0 0 0) 0x7f6f2c0b0610 con 0x7f6f2c0c5c50 2023-07-08T12:38:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:04.219 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:03.375667+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:04.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.218+0000 7f6f32a36700 1 -- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f0c06ad10 msgr2=0x7f6f0c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.218+0000 7f6f32a36700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f0c06ad10 0x7f6f0c06d1c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c000c00 tx=0x7f6f1c007000 comp rx=0 tx=0).stop 2023-07-08T12:38:04.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.218+0000 7f6f32a36700 1 -- 172.21.15.12:0/1826072611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f2c0c5c50 msgr2=0x7f6f2c0c2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.218+0000 7f6f32a36700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f2c0c5c50 0x7f6f2c0c2670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6f14011a30 tx=0x7f6f14011d40 comp rx=0 tx=0).stop 2023-07-08T12:38:04.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.218+0000 7f6f32a36700 1 -- 172.21.15.12:0/1826072611 shutdown_connections 2023-07-08T12:38:04.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.218+0000 7f6f32a36700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f0c06ad10 0x7f6f0c06d1c0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c000c00 tx=0x7f6f1c007000 comp rx=0 tx=0).stop 2023-07-08T12:38:04.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.218+0000 7f6f32a36700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f2c0b08f0 0x7f6f2c0b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.218+0000 7f6f32a36700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f18003a00 0x7f6f2c0af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.218+0000 7f6f32a36700 1 --2- 172.21.15.12:0/1826072611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f2c0c5c50 0x7f6f2c0c2670 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.218+0000 7f6f32a36700 1 -- 172.21.15.12:0/1826072611 >> 172.21.15.12:0/1826072611 conn(0x7f6f2c0cc290 msgr2=0x7f6f2c0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:04.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.218+0000 7f6f32a36700 1 -- 172.21.15.12:0/1826072611 shutdown_connections 2023-07-08T12:38:04.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.218+0000 7f6f32a36700 1 -- 172.21.15.12:0/1826072611 wait complete. 2023-07-08T12:38:04.233 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-08T12:38:04.233 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:38:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.358+0000 7f85024da700 1 Processor -- start 2023-07-08T12:38:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.358+0000 7f85024da700 1 -- start start 2023-07-08T12:38:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.359+0000 7f85024da700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f84fc06e0c0 0x7f84fc1608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.359+0000 7f85024da700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84fc0cc240 0x7f84fc0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.359+0000 7f85024da700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84fc160410 con 0x7f84fc156a10 2023-07-08T12:38:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.359+0000 7f85024da700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc0ccc00 con 0x7f84fc06e0c0 2023-07-08T12:38:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.359+0000 7f85024da700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc156de0 con 0x7f84fc0cc240 2023-07-08T12:38:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.359+0000 7f84fb7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f84fc06e0c0 0x7f84fc1608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.359+0000 7f84fb7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f84fc06e0c0 0x7f84fc1608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48950/0 (socket says 172.21.15.12:48950) 2023-07-08T12:38:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.359+0000 7f84fb7fe700 1 -- 172.21.15.12:0/1249840075 learned_addr learned my addr 172.21.15.12:0/1249840075 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.359+0000 7f8500a77700 1 --2- 172.21.15.12:0/1249840075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84fc0cc240 0x7f84fc0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.359+0000 7f84fb7fe700 1 -- 172.21.15.12:0/1249840075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84fc0cc240 msgr2=0x7f84fc0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.359+0000 7f84fb7fe700 1 --2- 172.21.15.12:0/1249840075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84fc0cc240 0x7f84fc0cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.360+0000 7f84fb7fe700 1 -- 172.21.15.12:0/1249840075 >> v1:172.21.15.160:6789/0 conn(0x7f84fc156a10 legacy=0x7f84fc15d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.360+0000 7f84fb7fe700 1 -- 172.21.15.12:0/1249840075 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc06ec70 con 0x7f84fc06e0c0 2023-07-08T12:38:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.360+0000 7f84fb7fe700 1 --2- 172.21.15.12:0/1249840075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc06e0c0 0x7f84fc1608a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0098f0 tx=0x7f84ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4148e491ac09d01f server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.360+0000 7f84faffd700 1 -- 172.21.15.12:0/1249840075 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec011830 con 0x7f84fc06e0c0 2023-07-08T12:38:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.360+0000 7f84faffd700 1 -- 172.21.15.12:0/1249840075 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec011990 con 0x7f84fc06e0c0 2023-07-08T12:38:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.360+0000 7f84faffd700 1 -- 172.21.15.12:0/1249840075 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec011d20 con 0x7f84fc06e0c0 2023-07-08T12:38:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.360+0000 7f85024da700 1 -- 172.21.15.12:0/1249840075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc06e0c0 msgr2=0x7f84fc1608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.360+0000 7f85024da700 1 --2- 172.21.15.12:0/1249840075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc06e0c0 0x7f84fc1608a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0098f0 tx=0x7f84ec009c00 comp rx=0 tx=0).stop 2023-07-08T12:38:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.360+0000 7f85024da700 1 -- 172.21.15.12:0/1249840075 shutdown_connections 2023-07-08T12:38:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.360+0000 7f85024da700 1 --2- 172.21.15.12:0/1249840075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84fc0cc240 0x7f84fc0cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.360+0000 7f85024da700 1 --2- 172.21.15.12:0/1249840075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc06e0c0 0x7f84fc1608a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.360+0000 7f85024da700 1 -- 172.21.15.12:0/1249840075 >> 172.21.15.12:0/1249840075 conn(0x7f84fc098b50 msgr2=0x7f84fc06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.361+0000 7f85024da700 1 -- 172.21.15.12:0/1249840075 shutdown_connections 2023-07-08T12:38:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.361+0000 7f85024da700 1 -- 172.21.15.12:0/1249840075 wait complete. 2023-07-08T12:38:04.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.362+0000 7f85024da700 1 Processor -- start 2023-07-08T12:38:04.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.362+0000 7f85024da700 1 -- start start 2023-07-08T12:38:04.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.363+0000 7f85024da700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84fc06e0c0 0x7f84fc151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.363+0000 7f85024da700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc0cc240 0x7f84fc157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.363+0000 7f85024da700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84fc156a10 0x7f84fc154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.363+0000 7f85024da700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc159e70 con 0x7f84fc0cc240 2023-07-08T12:38:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.363+0000 7f85024da700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc1f24d0 con 0x7f84fc06e0c0 2023-07-08T12:38:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.363+0000 7f85024da700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc158820 con 0x7f84fc156a10 2023-07-08T12:38:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.363+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc0cc240 0x7f84fc157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.363+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc0cc240 0x7f84fc157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48956/0 (socket says 172.21.15.12:48956) 2023-07-08T12:38:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.363+0000 7f84fb7fe700 1 -- 172.21.15.12:0/3736128282 learned_addr learned my addr 172.21.15.12:0/3736128282 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.363+0000 7f8500a77700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84fc156a10 0x7f84fc154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.363+0000 7f84fbfff700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84fc06e0c0 0x7f84fc151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.364+0000 7f84fb7fe700 1 -- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84fc156a10 msgr2=0x7f84fc154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.364+0000 7f84fb7fe700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84fc156a10 0x7f84fc154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.364+0000 7f84fb7fe700 1 -- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84fc06e0c0 msgr2=0x7f84fc151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.364+0000 7f84fb7fe700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84fc06e0c0 0x7f84fc151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.364+0000 7f84fb7fe700 1 -- 172.21.15.12:0/3736128282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec009550 con 0x7f84fc0cc240 2023-07-08T12:38:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.364+0000 7f84fb7fe700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc0cc240 0x7f84fc157df0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009c40 tx=0x7f84ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.364+0000 7f84f8ff9700 1 -- 172.21.15.12:0/3736128282 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec0045d0 con 0x7f84fc0cc240 2023-07-08T12:38:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.364+0000 7f84f8ff9700 1 -- 172.21.15.12:0/3736128282 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec01f070 con 0x7f84fc0cc240 2023-07-08T12:38:04.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.364+0000 7f85024da700 1 -- 172.21.15.12:0/3736128282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc1f5b90 con 0x7f84fc0cc240 2023-07-08T12:38:04.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.364+0000 7f84f8ff9700 1 -- 172.21.15.12:0/3736128282 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec0038a0 con 0x7f84fc0cc240 2023-07-08T12:38:04.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.364+0000 7f85024da700 1 -- 172.21.15.12:0/3736128282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc1f6150 con 0x7f84fc0cc240 2023-07-08T12:38:04.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.365+0000 7f85024da700 1 -- 172.21.15.12:0/3736128282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84fc04ea60 con 0x7f84fc0cc240 2023-07-08T12:38:04.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.365+0000 7f84f8ff9700 1 -- 172.21.15.12:0/3736128282 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f84ec003a00 con 0x7f84fc0cc240 2023-07-08T12:38:04.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.366+0000 7f84f8ff9700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f84dc06ace0 0x7f84dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.366+0000 7f84f8ff9700 1 -- 172.21.15.12:0/3736128282 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f84ec021070 con 0x7f84fc0cc240 2023-07-08T12:38:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.366+0000 7f84fbfff700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f84dc06ace0 0x7f84dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.366+0000 7f84fbfff700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f84dc06ace0 0x7f84dc06d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84fc15dbd0 tx=0x7f84e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:04.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.370+0000 7f84f8ff9700 1 -- 172.21.15.12:0/3736128282 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f84ec055e90 con 0x7f84fc0cc240 2023-07-08T12:38:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.567+0000 7f85024da700 1 -- 172.21.15.12:0/3736128282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84fc0ccde0 con 0x7f84fc0cc240 2023-07-08T12:38:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.567+0000 7f84f8ff9700 1 -- 172.21.15.12:0/3736128282 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84ec003cb0 con 0x7f84fc0cc240 2023-07-08T12:38:04.570 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:38:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.569+0000 7f85024da700 1 -- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f84dc06ace0 msgr2=0x7f84dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.569+0000 7f85024da700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f84dc06ace0 0x7f84dc06d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84fc15dbd0 tx=0x7f84e4009040 comp rx=0 tx=0).stop 2023-07-08T12:38:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.569+0000 7f85024da700 1 -- 172.21.15.12:0/3736128282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc0cc240 msgr2=0x7f84fc157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.569+0000 7f85024da700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc0cc240 0x7f84fc157df0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009c40 tx=0x7f84ec004750 comp rx=0 tx=0).stop 2023-07-08T12:38:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.569+0000 7f85024da700 1 -- 172.21.15.12:0/3736128282 shutdown_connections 2023-07-08T12:38:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.569+0000 7f85024da700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f84dc06ace0 0x7f84dc06d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.569+0000 7f85024da700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84fc156a10 0x7f84fc154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.569+0000 7f85024da700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84fc06e0c0 0x7f84fc151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.569+0000 7f85024da700 1 --2- 172.21.15.12:0/3736128282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc0cc240 0x7f84fc157df0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.569+0000 7f85024da700 1 -- 172.21.15.12:0/3736128282 >> 172.21.15.12:0/3736128282 conn(0x7f84fc098b50 msgr2=0x7f84fc06cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.569+0000 7f85024da700 1 -- 172.21.15.12:0/3736128282 shutdown_connections 2023-07-08T12:38:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.569+0000 7f85024da700 1 -- 172.21.15.12:0/3736128282 wait complete. 2023-07-08T12:38:04.584 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:38:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.709+0000 7f6f67a21700 1 Processor -- start 2023-07-08T12:38:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.709+0000 7f6f67a21700 1 -- start start 2023-07-08T12:38:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.710+0000 7f6f67a21700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6f60156a10 0x7f6f6006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.710+0000 7f6f67a21700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6f6015d390 0x7f6f601608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.710+0000 7f6f67a21700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f60160410 con 0x7f6f600cc240 2023-07-08T12:38:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.710+0000 7f6f67a21700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f60156de0 con 0x7f6f6015d390 2023-07-08T12:38:04.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.710+0000 7f6f67a21700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6006ec70 con 0x7f6f60156a10 2023-07-08T12:38:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.710+0000 7f6f64fbc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6f6015d390 0x7f6f601608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.710+0000 7f6f65fbe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6f600cc240 0x7f6f600ae440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36362/0 (socket says 172.21.15.12:36362) 2023-07-08T12:38:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.710+0000 7f6f65fbe700 1 -- 172.21.15.12:0/3630971135 learned_addr learned my addr 172.21.15.12:0/3630971135 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.710+0000 7f6f657bd700 1 --2- 172.21.15.12:0/3630971135 >> v2:172.21.15.160:3300/0 conn(0x7f6f60156a10 0x7f6f6006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f64fbc700 1 -- 172.21.15.12:0/3630971135 >> v2:172.21.15.160:3300/0 conn(0x7f6f60156a10 msgr2=0x7f6f6006e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f64fbc700 1 --2- 172.21.15.12:0/3630971135 >> v2:172.21.15.160:3300/0 conn(0x7f6f60156a10 0x7f6f6006e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f64fbc700 1 -- 172.21.15.12:0/3630971135 >> v1:172.21.15.12:6789/0 conn(0x7f6f600cc240 legacy=0x7f6f600ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f64fbc700 1 -- 172.21.15.12:0/3630971135 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6006e6b0 con 0x7f6f6015d390 2023-07-08T12:38:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f57fff700 1 -- 172.21.15.12:0/3630971135 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3191032858 0 0) 0x7f6f60160410 con 0x7f6f600cc240 2023-07-08T12:38:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f657bd700 1 --2- 172.21.15.12:0/3630971135 >> v2:172.21.15.160:3300/0 conn(0x7f6f60156a10 0x7f6f6006e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f64fbc700 1 --2- 172.21.15.12:0/3630971135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f6015d390 0x7f6f601608a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6f60072490 tx=0x7f6f5c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a325b3ab73d9a85 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f57fff700 1 -- 172.21.15.12:0/3630971135 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c011830 con 0x7f6f6015d390 2023-07-08T12:38:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f57fff700 1 -- 172.21.15.12:0/3630971135 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c011990 con 0x7f6f6015d390 2023-07-08T12:38:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f57fff700 1 -- 172.21.15.12:0/3630971135 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c011d20 con 0x7f6f6015d390 2023-07-08T12:38:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f67a21700 1 -- 172.21.15.12:0/3630971135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f6015d390 msgr2=0x7f6f601608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f67a21700 1 --2- 172.21.15.12:0/3630971135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f6015d390 0x7f6f601608a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6f60072490 tx=0x7f6f5c009b20 comp rx=0 tx=0).stop 2023-07-08T12:38:04.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f67a21700 1 -- 172.21.15.12:0/3630971135 shutdown_connections 2023-07-08T12:38:04.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f67a21700 1 --2- 172.21.15.12:0/3630971135 >> v2:172.21.15.160:3300/0 conn(0x7f6f60156a10 0x7f6f6006e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f67a21700 1 --2- 172.21.15.12:0/3630971135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f6015d390 0x7f6f601608a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.711+0000 7f6f67a21700 1 -- 172.21.15.12:0/3630971135 >> 172.21.15.12:0/3630971135 conn(0x7f6f60098b50 msgr2=0x7f6f6006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:04.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.712+0000 7f6f67a21700 1 -- 172.21.15.12:0/3630971135 shutdown_connections 2023-07-08T12:38:04.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.712+0000 7f6f67a21700 1 -- 172.21.15.12:0/3630971135 wait complete. 2023-07-08T12:38:04.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.713+0000 7f6f67a21700 1 Processor -- start 2023-07-08T12:38:04.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.713+0000 7f6f67a21700 1 -- start start 2023-07-08T12:38:04.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.714+0000 7f6f67a21700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f600cc240 0x7f6f60154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.714+0000 7f6f67a21700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f60156a10 0x7f6f6009ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.714+0000 7f6f67a21700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f6015d390 0x7f6f6009b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.714+0000 7f6f67a21700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6009d370 con 0x7f6f6015d390 2023-07-08T12:38:04.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.714+0000 7f6f67a21700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6009a5d0 con 0x7f6f600cc240 2023-07-08T12:38:04.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.714+0000 7f6f67a21700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6009a8b0 con 0x7f6f60156a10 2023-07-08T12:38:04.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.714+0000 7f6f65fbe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f6015d390 0x7f6f6009b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.714+0000 7f6f65fbe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f6015d390 0x7f6f6009b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48972/0 (socket says 172.21.15.12:48972) 2023-07-08T12:38:04.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.714+0000 7f6f65fbe700 1 -- 172.21.15.12:0/2786785054 learned_addr learned my addr 172.21.15.12:0/2786785054 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:04.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.714+0000 7f6f657bd700 1 --2- 172.21.15.12:0/2786785054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f600cc240 0x7f6f60154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.715+0000 7f6f65fbe700 1 -- 172.21.15.12:0/2786785054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f60156a10 msgr2=0x7f6f6009ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.715+0000 7f6f65fbe700 1 --2- 172.21.15.12:0/2786785054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f60156a10 0x7f6f6009ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.715+0000 7f6f65fbe700 1 -- 172.21.15.12:0/2786785054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f600cc240 msgr2=0x7f6f60154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.715+0000 7f6f65fbe700 1 --2- 172.21.15.12:0/2786785054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f600cc240 0x7f6f60154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.715+0000 7f6f65fbe700 1 -- 172.21.15.12:0/2786785054 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c0094a0 con 0x7f6f6015d390 2023-07-08T12:38:04.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.715+0000 7f6f65fbe700 1 --2- 172.21.15.12:0/2786785054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f6015d390 0x7f6f6009b670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800cba0 tx=0x7f6f5800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:04.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.715+0000 7f6f55ffb700 1 -- 172.21.15.12:0/2786785054 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800ab80 con 0x7f6f6015d390 2023-07-08T12:38:04.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.715+0000 7f6f55ffb700 1 -- 172.21.15.12:0/2786785054 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5800ad00 con 0x7f6f6015d390 2023-07-08T12:38:04.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.715+0000 7f6f67a21700 1 -- 172.21.15.12:0/2786785054 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6009ab90 con 0x7f6f6015d390 2023-07-08T12:38:04.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.715+0000 7f6f55ffb700 1 -- 172.21.15.12:0/2786785054 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f58007310 con 0x7f6f6015d390 2023-07-08T12:38:04.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.715+0000 7f6f67a21700 1 -- 172.21.15.12:0/2786785054 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6009b150 con 0x7f6f6015d390 2023-07-08T12:38:04.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.716+0000 7f6f67a21700 1 -- 172.21.15.12:0/2786785054 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6009b340 con 0x7f6f6015d390 2023-07-08T12:38:04.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.716+0000 7f6f55ffb700 1 -- 172.21.15.12:0/2786785054 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6f58007470 con 0x7f6f6015d390 2023-07-08T12:38:04.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.717+0000 7f6f55ffb700 1 --2- 172.21.15.12:0/2786785054 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f4406ace0 0x7f6f4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:04.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.717+0000 7f6f55ffb700 1 -- 172.21.15.12:0/2786785054 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6f58088f30 con 0x7f6f6015d390 2023-07-08T12:38:04.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.717+0000 7f6f657bd700 1 --2- 172.21.15.12:0/2786785054 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f4406ace0 0x7f6f4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:04.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.717+0000 7f6f657bd700 1 --2- 172.21.15.12:0/2786785054 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f4406ace0 0x7f6f4406d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000c920 tx=0x7f6f50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:04.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.721+0000 7f6f55ffb700 1 -- 172.21.15.12:0/2786785054 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6f580543c0 con 0x7f6f6015d390 2023-07-08T12:38:04.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.918+0000 7f6f67a21700 1 -- 172.21.15.12:0/2786785054 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6f6004ea60 con 0x7f6f6015d390 2023-07-08T12:38:04.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.919+0000 7f6f55ffb700 1 -- 172.21.15.12:0/2786785054 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6f58018020 con 0x7f6f6015d390 2023-07-08T12:38:04.921 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:38:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.921+0000 7f6f67a21700 1 -- 172.21.15.12:0/2786785054 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f4406ace0 msgr2=0x7f6f4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.921+0000 7f6f67a21700 1 --2- 172.21.15.12:0/2786785054 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f4406ace0 0x7f6f4406d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000c920 tx=0x7f6f50008040 comp rx=0 tx=0).stop 2023-07-08T12:38:04.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.921+0000 7f6f67a21700 1 -- 172.21.15.12:0/2786785054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f6015d390 msgr2=0x7f6f6009b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:04.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.921+0000 7f6f67a21700 1 --2- 172.21.15.12:0/2786785054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f6015d390 0x7f6f6009b670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800cba0 tx=0x7f6f5800cf60 comp rx=0 tx=0).stop 2023-07-08T12:38:04.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.921+0000 7f6f67a21700 1 -- 172.21.15.12:0/2786785054 shutdown_connections 2023-07-08T12:38:04.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.921+0000 7f6f67a21700 1 --2- 172.21.15.12:0/2786785054 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f4406ace0 0x7f6f4406d190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.921+0000 7f6f67a21700 1 --2- 172.21.15.12:0/2786785054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f6015d390 0x7f6f6009b670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.921+0000 7f6f67a21700 1 --2- 172.21.15.12:0/2786785054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f600cc240 0x7f6f60154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.921+0000 7f6f67a21700 1 --2- 172.21.15.12:0/2786785054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f60156a10 0x7f6f6009ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.921+0000 7f6f67a21700 1 -- 172.21.15.12:0/2786785054 >> 172.21.15.12:0/2786785054 conn(0x7f6f60098b50 msgr2=0x7f6f600ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.921+0000 7f6f67a21700 1 -- 172.21.15.12:0/2786785054 shutdown_connections 2023-07-08T12:38:04.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:04.921+0000 7f6f67a21700 1 -- 172.21.15.12:0/2786785054 wait complete. 2023-07-08T12:38:04.936 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-08T12:38:05.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.059+0000 7f634359d700 1 Processor -- start 2023-07-08T12:38:05.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.059+0000 7f634359d700 1 -- start start 2023-07-08T12:38:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.060+0000 7f634359d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f633c15a480 0x7f633c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.060+0000 7f634359d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633c0b2410 0x7f633c0b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.060+0000 7f634359d700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f633c15def0 0x7f633c15e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.060+0000 7f634359d700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633c15a850 con 0x7f633c15a480 2023-07-08T12:38:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.060+0000 7f634359d700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633c0b2da0 con 0x7f633c15def0 2023-07-08T12:38:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.060+0000 7f634359d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0cde90 con 0x7f633c0b2410 2023-07-08T12:38:05.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.060+0000 7f6341339700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f633c15a480 0x7f633c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:05.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.060+0000 7f6341339700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f633c15a480 0x7f633c0cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48976/0 (socket says 172.21.15.12:48976) 2023-07-08T12:38:05.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.060+0000 7f6341339700 1 -- 172.21.15.12:0/3610618741 learned_addr learned my addr 172.21.15.12:0/3610618741 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:05.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.060+0000 7f6341b3a700 1 --2- 172.21.15.12:0/3610618741 >> v2:172.21.15.160:3300/0 conn(0x7f633c15def0 0x7f633c15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:05.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.060+0000 7f6340b38700 1 --2- 172.21.15.12:0/3610618741 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633c0b2410 0x7f633c0b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:05.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.060+0000 7f6341339700 1 -- 172.21.15.12:0/3610618741 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633c0b2410 msgr2=0x7f633c0b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:05.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f6341339700 1 --2- 172.21.15.12:0/3610618741 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633c0b2410 0x7f633c0b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f6341339700 1 -- 172.21.15.12:0/3610618741 >> v2:172.21.15.160:3300/0 conn(0x7f633c15def0 msgr2=0x7f633c15e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:05.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f6341339700 1 --2- 172.21.15.12:0/3610618741 >> v2:172.21.15.160:3300/0 conn(0x7f633c15def0 0x7f633c15e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f6341339700 1 -- 172.21.15.12:0/3610618741 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c0cd8d0 con 0x7f633c15a480 2023-07-08T12:38:05.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f6341b3a700 1 --2- 172.21.15.12:0/3610618741 >> v2:172.21.15.160:3300/0 conn(0x7f633c15def0 0x7f633c15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:05.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f6341339700 1 --2- 172.21.15.12:0/3610618741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c15a480 0x7f633c0cafe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f633c0af8e0 tx=0x7f632c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a987010b16d4da server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:05.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f6333fff700 1 -- 172.21.15.12:0/3610618741 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00b870 con 0x7f633c15a480 2023-07-08T12:38:05.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f6333fff700 1 -- 172.21.15.12:0/3610618741 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c00b9d0 con 0x7f633c15a480 2023-07-08T12:38:05.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f6333fff700 1 -- 172.21.15.12:0/3610618741 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00bd60 con 0x7f633c15a480 2023-07-08T12:38:05.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f634359d700 1 -- 172.21.15.12:0/3610618741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c15a480 msgr2=0x7f633c0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:05.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f634359d700 1 --2- 172.21.15.12:0/3610618741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c15a480 0x7f633c0cafe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f633c0af8e0 tx=0x7f632c009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f634359d700 1 -- 172.21.15.12:0/3610618741 shutdown_connections 2023-07-08T12:38:05.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f634359d700 1 --2- 172.21.15.12:0/3610618741 >> v2:172.21.15.160:3300/0 conn(0x7f633c15def0 0x7f633c15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f634359d700 1 --2- 172.21.15.12:0/3610618741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c15a480 0x7f633c0cafe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f634359d700 1 --2- 172.21.15.12:0/3610618741 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633c0b2410 0x7f633c0b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.061+0000 7f634359d700 1 -- 172.21.15.12:0/3610618741 >> 172.21.15.12:0/3610618741 conn(0x7f633c06ab60 msgr2=0x7f633c06e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.062+0000 7f634359d700 1 -- 172.21.15.12:0/3610618741 shutdown_connections 2023-07-08T12:38:05.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.062+0000 7f634359d700 1 -- 172.21.15.12:0/3610618741 wait complete. 2023-07-08T12:38:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.063+0000 7f634359d700 1 Processor -- start 2023-07-08T12:38:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.063+0000 7f634359d700 1 -- start start 2023-07-08T12:38:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f634359d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633c0b2410 0x7f633c0af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f634359d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c15a480 0x7f633c06bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:05.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f634359d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f633c15def0 0x7f633c1512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f634359d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c153b30 con 0x7f633c15a480 2023-07-08T12:38:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f634359d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c153850 con 0x7f633c15def0 2023-07-08T12:38:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f634359d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c06b170 con 0x7f633c0b2410 2023-07-08T12:38:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6340b38700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c15a480 0x7f633c06bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6340b38700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c15a480 0x7f633c06bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48982/0 (socket says 172.21.15.12:48982) 2023-07-08T12:38:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6340b38700 1 -- 172.21.15.12:0/3820368298 learned_addr learned my addr 172.21.15.12:0/3820368298 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:05.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6341b3a700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f633c15def0 0x7f633c1512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6340b38700 1 -- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633c0b2410 msgr2=0x7f633c0af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6340b38700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633c0b2410 0x7f633c0af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6340b38700 1 -- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f633c15def0 msgr2=0x7f633c1512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6340b38700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f633c15def0 0x7f633c1512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6340b38700 1 -- 172.21.15.12:0/3820368298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c009550 con 0x7f633c15a480 2023-07-08T12:38:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6341b3a700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f633c15def0 0x7f633c1512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:05.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6340b38700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c15a480 0x7f633c06bc50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f633800c7f0 tx=0x7f633800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6331ffb700 1 -- 172.21.15.12:0/3820368298 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6338011800 con 0x7f633c15a480 2023-07-08T12:38:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6331ffb700 1 -- 172.21.15.12:0/3820368298 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6338011960 con 0x7f633c15a480 2023-07-08T12:38:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f634359d700 1 -- 172.21.15.12:0/3820368298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c06b450 con 0x7f633c15a480 2023-07-08T12:38:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.064+0000 7f6331ffb700 1 -- 172.21.15.12:0/3820368298 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6338011d10 con 0x7f633c15a480 2023-07-08T12:38:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.065+0000 7f634359d700 1 -- 172.21.15.12:0/3820368298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c15ab40 con 0x7f633c15a480 2023-07-08T12:38:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.066+0000 7f634359d700 1 -- 172.21.15.12:0/3820368298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c06b970 con 0x7f633c15a480 2023-07-08T12:38:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.066+0000 7f6331ffb700 1 -- 172.21.15.12:0/3820368298 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6338018360 con 0x7f633c15a480 2023-07-08T12:38:05.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.066+0000 7f6331ffb700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f632006ace0 0x7f632006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.066+0000 7f6331ffb700 1 -- 172.21.15.12:0/3820368298 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6338088bf0 con 0x7f633c15a480 2023-07-08T12:38:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.067+0000 7f6341339700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f632006ace0 0x7f632006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.067+0000 7f6341339700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f632006ace0 0x7f632006d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f632c009c10 tx=0x7f632c010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:05.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.071+0000 7f6331ffb700 1 -- 172.21.15.12:0/3820368298 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f633c06b970 con 0x7f633c15a480 2023-07-08T12:38:05.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.271+0000 7f634359d700 1 -- 172.21.15.12:0/3820368298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f633c06b970 con 0x7f633c15a480 2023-07-08T12:38:05.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.272+0000 7f6331ffb700 1 -- 172.21.15.12:0/3820368298 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+126 (secure 0 0 0) 0x7f633c06b970 con 0x7f633c15a480 2023-07-08T12:38:05.274 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:05.275 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2023-07-08T12:38:05.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:38:05.275 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 1, 2023-07-08T12:38:05.275 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": ".mgr" 2023-07-08T12:38:05.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:38:05.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:38:05.275 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 2, 2023-07-08T12:38:05.275 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": "rbd" 2023-07-08T12:38:05.276 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:38:05.276 INFO:teuthology.orchestra.run.smithi012.stdout:] 2023-07-08T12:38:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.274+0000 7f634359d700 1 -- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f632006ace0 msgr2=0x7f632006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.274+0000 7f634359d700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f632006ace0 0x7f632006d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f632c009c10 tx=0x7f632c010000 comp rx=0 tx=0).stop 2023-07-08T12:38:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.274+0000 7f634359d700 1 -- 172.21.15.12:0/3820368298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c15a480 msgr2=0x7f633c06bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.274+0000 7f634359d700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c15a480 0x7f633c06bc50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f633800c7f0 tx=0x7f633800cbb0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.274+0000 7f634359d700 1 -- 172.21.15.12:0/3820368298 shutdown_connections 2023-07-08T12:38:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.274+0000 7f634359d700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f632006ace0 0x7f632006d190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.274+0000 7f634359d700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f633c15def0 0x7f633c1512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.274+0000 7f634359d700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633c0b2410 0x7f633c0af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.274+0000 7f634359d700 1 --2- 172.21.15.12:0/3820368298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c15a480 0x7f633c06bc50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.274+0000 7f634359d700 1 -- 172.21.15.12:0/3820368298 >> 172.21.15.12:0/3820368298 conn(0x7f633c06ab60 msgr2=0x7f633c0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.274+0000 7f634359d700 1 -- 172.21.15.12:0/3820368298 shutdown_connections 2023-07-08T12:38:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.274+0000 7f634359d700 1 -- 172.21.15.12:0/3820368298 wait complete. 2023-07-08T12:38:05.288 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-08T12:38:05.288 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-08T12:38:05.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.414+0000 7f125ccb7700 1 Processor -- start 2023-07-08T12:38:05.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.414+0000 7f125ccb7700 1 -- start start 2023-07-08T12:38:05.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.415+0000 7f125ccb7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1258159ff0 0x7f12580cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:05.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.415+0000 7f125ccb7700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f125815d5d0 0x7f1258069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:05.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.415+0000 7f125ccb7700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12580715c0 con 0x7f125806a470 2023-07-08T12:38:05.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.415+0000 7f125ccb7700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12580725d0 con 0x7f1258159ff0 2023-07-08T12:38:05.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.415+0000 7f125ccb7700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125806a870 con 0x7f125815d5d0 2023-07-08T12:38:05.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.415+0000 7f1256d9d700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f125806a470 0x7f125815fea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36388/0 (socket says 172.21.15.12:36388) 2023-07-08T12:38:05.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.415+0000 7f125659c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1258159ff0 0x7f12580cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:05.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.415+0000 7f1256d9d700 1 -- 172.21.15.12:0/3372943115 learned_addr learned my addr 172.21.15.12:0/3372943115 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:05.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.415+0000 7f1255d9b700 1 --2- 172.21.15.12:0/3372943115 >> v2:172.21.15.160:3300/0 conn(0x7f125815d5d0 0x7f1258069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:05.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.416+0000 7f125559a700 1 -- 172.21.15.12:0/3372943115 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579670278 0 0) 0x7f12580715c0 con 0x7f125806a470 2023-07-08T12:38:05.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.416+0000 7f125559a700 1 -- 172.21.15.12:0/3372943115 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12400035b0 con 0x7f125806a470 2023-07-08T12:38:05.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.416+0000 7f125659c700 1 -- 172.21.15.12:0/3372943115 >> v2:172.21.15.160:3300/0 conn(0x7f125815d5d0 msgr2=0x7f1258069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:05.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.416+0000 7f125659c700 1 --2- 172.21.15.12:0/3372943115 >> v2:172.21.15.160:3300/0 conn(0x7f125815d5d0 0x7f1258069f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.416+0000 7f125659c700 1 -- 172.21.15.12:0/3372943115 >> v1:172.21.15.12:6789/0 conn(0x7f125806a470 legacy=0x7f125815fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:05.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.416+0000 7f125659c700 1 -- 172.21.15.12:0/3372943115 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1258071810 con 0x7f1258159ff0 2023-07-08T12:38:05.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.416+0000 7f125659c700 1 --2- 172.21.15.12:0/3372943115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1258159ff0 0x7f12580cd4e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12580b24e0 tx=0x7f124c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4586168fb683d3e server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:05.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.416+0000 7f125559a700 1 -- 172.21.15.12:0/3372943115 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c011830 con 0x7f1258159ff0 2023-07-08T12:38:05.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.416+0000 7f125559a700 1 -- 172.21.15.12:0/3372943115 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c011990 con 0x7f1258159ff0 2023-07-08T12:38:05.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.416+0000 7f125559a700 1 -- 172.21.15.12:0/3372943115 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c011d20 con 0x7f1258159ff0 2023-07-08T12:38:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.417+0000 7f125ccb7700 1 -- 172.21.15.12:0/3372943115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1258159ff0 msgr2=0x7f12580cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.417+0000 7f125ccb7700 1 --2- 172.21.15.12:0/3372943115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1258159ff0 0x7f12580cd4e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12580b24e0 tx=0x7f124c009b20 comp rx=0 tx=0).stop 2023-07-08T12:38:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.417+0000 7f125ccb7700 1 -- 172.21.15.12:0/3372943115 shutdown_connections 2023-07-08T12:38:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.417+0000 7f125ccb7700 1 --2- 172.21.15.12:0/3372943115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1258159ff0 0x7f12580cd4e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.417+0000 7f125ccb7700 1 --2- 172.21.15.12:0/3372943115 >> v2:172.21.15.160:3300/0 conn(0x7f125815d5d0 0x7f1258069f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.417+0000 7f125ccb7700 1 -- 172.21.15.12:0/3372943115 >> 172.21.15.12:0/3372943115 conn(0x7f125806fd80 msgr2=0x7f12580b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.417+0000 7f125ccb7700 1 -- 172.21.15.12:0/3372943115 shutdown_connections 2023-07-08T12:38:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.417+0000 7f125ccb7700 1 -- 172.21.15.12:0/3372943115 wait complete. 2023-07-08T12:38:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.418+0000 7f125ccb7700 1 Processor -- start 2023-07-08T12:38:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.418+0000 7f125ccb7700 1 -- start start 2023-07-08T12:38:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f125ccb7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f125806a470 0x7f125814df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f125ccb7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1258159ff0 0x7f125814ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f125ccb7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f125815d5d0 0x7f12580adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f125ccb7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12580b5730 con 0x7f125815d5d0 2023-07-08T12:38:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f125ccb7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12580adaa0 con 0x7f125806a470 2023-07-08T12:38:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f125ccb7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125814e460 con 0x7f1258159ff0 2023-07-08T12:38:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f1256d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f125815d5d0 0x7f12580adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f1256d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f125815d5d0 0x7f12580adf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48994/0 (socket says 172.21.15.12:48994) 2023-07-08T12:38:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f1256d9d700 1 -- 172.21.15.12:0/1811263882 learned_addr learned my addr 172.21.15.12:0/1811263882 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f125659c700 1 --2- 172.21.15.12:0/1811263882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f125806a470 0x7f125814df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f1256d9d700 1 -- 172.21.15.12:0/1811263882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1258159ff0 msgr2=0x7f125814ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f1256d9d700 1 --2- 172.21.15.12:0/1811263882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1258159ff0 0x7f125814ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f1256d9d700 1 -- 172.21.15.12:0/1811263882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f125806a470 msgr2=0x7f125814df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f1256d9d700 1 --2- 172.21.15.12:0/1811263882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f125806a470 0x7f125814df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f1256d9d700 1 -- 172.21.15.12:0/1811263882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0094a0 con 0x7f125815d5d0 2023-07-08T12:38:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f1256d9d700 1 --2- 172.21.15.12:0/1811263882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f125815d5d0 0x7f12580adf30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f125000caa0 tx=0x7f125000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f1246ffd700 1 -- 172.21.15.12:0/1811263882 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125000aa80 con 0x7f125815d5d0 2023-07-08T12:38:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.419+0000 7f1246ffd700 1 -- 172.21.15.12:0/1811263882 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125000abe0 con 0x7f125815d5d0 2023-07-08T12:38:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.420+0000 7f125ccb7700 1 -- 172.21.15.12:0/1811263882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12580acbb0 con 0x7f125815d5d0 2023-07-08T12:38:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.420+0000 7f1246ffd700 1 -- 172.21.15.12:0/1811263882 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250012400 con 0x7f125815d5d0 2023-07-08T12:38:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.420+0000 7f125ccb7700 1 -- 172.21.15.12:0/1811263882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12580ad170 con 0x7f125815d5d0 2023-07-08T12:38:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.420+0000 7f125ccb7700 1 -- 172.21.15.12:0/1811263882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1258074180 con 0x7f125815d5d0 2023-07-08T12:38:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.421+0000 7f1246ffd700 1 -- 172.21.15.12:0/1811263882 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f125001b3b0 con 0x7f125815d5d0 2023-07-08T12:38:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.421+0000 7f1246ffd700 1 --2- 172.21.15.12:0/1811263882 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f123806ace0 0x7f123806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.421+0000 7f1246ffd700 1 -- 172.21.15.12:0/1811263882 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1250088e70 con 0x7f125815d5d0 2023-07-08T12:38:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.422+0000 7f125659c700 1 --2- 172.21.15.12:0/1811263882 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f123806ace0 0x7f123806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.422+0000 7f125659c700 1 --2- 172.21.15.12:0/1811263882 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f123806ace0 0x7f123806d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f124c0090b0 tx=0x7f124c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.426+0000 7f1246ffd700 1 -- 172.21.15.12:0/1811263882 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1250054270 con 0x7f125815d5d0 2023-07-08T12:38:05.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:05.629+0000 7f125ccb7700 1 -- 172.21.15.12:0/1811263882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f12580adbe0 con 0x7f125815d5d0 2023-07-08T12:38:05.795 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:38:05.799 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:38:06.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.517+0000 7f1246ffd700 1 -- 172.21.15.12:0/1811263882 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v21) v1 ==== 138+0+0 (secure 0 0 0) 0x7f12500170b0 con 0x7f125815d5d0 2023-07-08T12:38:06.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.520+0000 7f125ccb7700 1 -- 172.21.15.12:0/1811263882 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f123806ace0 msgr2=0x7f123806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:06.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.520+0000 7f125ccb7700 1 --2- 172.21.15.12:0/1811263882 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f123806ace0 0x7f123806d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f124c0090b0 tx=0x7f124c008030 comp rx=0 tx=0).stop 2023-07-08T12:38:06.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.520+0000 7f125ccb7700 1 -- 172.21.15.12:0/1811263882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f125815d5d0 msgr2=0x7f12580adf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:06.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.520+0000 7f125ccb7700 1 --2- 172.21.15.12:0/1811263882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f125815d5d0 0x7f12580adf30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f125000caa0 tx=0x7f125000ce60 comp rx=0 tx=0).stop 2023-07-08T12:38:06.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.520+0000 7f125ccb7700 1 -- 172.21.15.12:0/1811263882 shutdown_connections 2023-07-08T12:38:06.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.520+0000 7f125ccb7700 1 --2- 172.21.15.12:0/1811263882 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f123806ace0 0x7f123806d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:06.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.520+0000 7f125ccb7700 1 --2- 172.21.15.12:0/1811263882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f125815d5d0 0x7f12580adf30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:06.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.520+0000 7f125ccb7700 1 --2- 172.21.15.12:0/1811263882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f125806a470 0x7f125814df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:06.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.520+0000 7f125ccb7700 1 --2- 172.21.15.12:0/1811263882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1258159ff0 0x7f125814ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:06.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.520+0000 7f125ccb7700 1 -- 172.21.15.12:0/1811263882 >> 172.21.15.12:0/1811263882 conn(0x7f125806fd80 msgr2=0x7f125815db00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.522+0000 7f125ccb7700 1 -- 172.21.15.12:0/1811263882 shutdown_connections 2023-07-08T12:38:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.522+0000 7f125ccb7700 1 -- 172.21.15.12:0/1811263882 wait complete. 2023-07-08T12:38:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' created 2023-07-08T12:38:06.542 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-07-08T12:38:06.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.679+0000 7faebd738700 1 Processor -- start 2023-07-08T12:38:06.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.679+0000 7faebd738700 1 -- start start 2023-07-08T12:38:06.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.680+0000 7faebd738700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faeb80cd5d0 0x7faeb815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.680+0000 7faebd738700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faeb8070ae0 0x7faeb8070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.680+0000 7faebd738700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb80cc910 con 0x7faeb8158a00 2023-07-08T12:38:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.680+0000 7faebd738700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb80714a0 con 0x7faeb80cd5d0 2023-07-08T12:38:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.680+0000 7faebd738700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb8069f30 con 0x7faeb8070ae0 2023-07-08T12:38:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.681+0000 7faeb67fc700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faeb80cd5d0 0x7faeb815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.681+0000 7faeb77fe700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faeb8070ae0 0x7faeb8070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.681+0000 7faeb6ffd700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7faeb8158a00 0x7faeb806afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:45620/0 (socket says 172.21.15.12:45620) 2023-07-08T12:38:06.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.681+0000 7faeb6ffd700 1 -- 172.21.15.12:0/1104129170 learned_addr learned my addr 172.21.15.12:0/1104129170 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:06.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.681+0000 7faeb77fe700 1 -- 172.21.15.12:0/1104129170 >> v2:172.21.15.160:3300/0 conn(0x7faeb80cd5d0 msgr2=0x7faeb815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:06.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.681+0000 7faeb77fe700 1 --2- 172.21.15.12:0/1104129170 >> v2:172.21.15.160:3300/0 conn(0x7faeb80cd5d0 0x7faeb815cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:06.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.681+0000 7faeb77fe700 1 -- 172.21.15.12:0/1104129170 >> v1:172.21.15.160:6789/0 conn(0x7faeb8158a00 legacy=0x7faeb806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:06.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.681+0000 7faeb77fe700 1 -- 172.21.15.12:0/1104129170 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb806a210 con 0x7faeb8070ae0 2023-07-08T12:38:06.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.681+0000 7faeb67fc700 1 --2- 172.21.15.12:0/1104129170 >> v2:172.21.15.160:3300/0 conn(0x7faeb80cd5d0 0x7faeb815cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:38:06.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.681+0000 7faeb77fe700 1 --2- 172.21.15.12:0/1104129170 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faeb8070ae0 0x7faeb8070f60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faeb000aea0 tx=0x7faeb000c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70751b766ee94f5a server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:06.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.681+0000 7faeb5ffb700 1 -- 172.21.15.12:0/1104129170 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb000f070 con 0x7faeb8070ae0 2023-07-08T12:38:06.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.681+0000 7faeb5ffb700 1 -- 172.21.15.12:0/1104129170 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb000a2e0 con 0x7faeb8070ae0 2023-07-08T12:38:06.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.681+0000 7faeb5ffb700 1 -- 172.21.15.12:0/1104129170 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb000cb50 con 0x7faeb8070ae0 2023-07-08T12:38:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.682+0000 7faebd738700 1 -- 172.21.15.12:0/1104129170 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faeb8070ae0 msgr2=0x7faeb8070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.682+0000 7faebd738700 1 --2- 172.21.15.12:0/1104129170 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faeb8070ae0 0x7faeb8070f60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faeb000aea0 tx=0x7faeb000c330 comp rx=0 tx=0).stop 2023-07-08T12:38:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.682+0000 7faebd738700 1 -- 172.21.15.12:0/1104129170 shutdown_connections 2023-07-08T12:38:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.682+0000 7faebd738700 1 --2- 172.21.15.12:0/1104129170 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faeb8070ae0 0x7faeb8070f60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.682+0000 7faebd738700 1 --2- 172.21.15.12:0/1104129170 >> v2:172.21.15.160:3300/0 conn(0x7faeb80cd5d0 0x7faeb815cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.682+0000 7faebd738700 1 -- 172.21.15.12:0/1104129170 >> 172.21.15.12:0/1104129170 conn(0x7faeb80cf1d0 msgr2=0x7faeb80cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.682+0000 7faebd738700 1 -- 172.21.15.12:0/1104129170 shutdown_connections 2023-07-08T12:38:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.682+0000 7faebd738700 1 -- 172.21.15.12:0/1104129170 wait complete. 2023-07-08T12:38:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.683+0000 7faebd738700 1 Processor -- start 2023-07-08T12:38:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.683+0000 7faebd738700 1 -- start start 2023-07-08T12:38:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.684+0000 7faebd738700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faeb8070ae0 0x7faeb815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.684+0000 7faebd738700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb80cd5d0 0x7faeb815b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.684+0000 7faebd738700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faeb8158a00 0x7faeb81e3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.684+0000 7faebd738700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb8149d70 con 0x7faeb80cd5d0 2023-07-08T12:38:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.684+0000 7faebd738700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb81e4bd0 con 0x7faeb8070ae0 2023-07-08T12:38:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.684+0000 7faebd738700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb81e4eb0 con 0x7faeb8158a00 2023-07-08T12:38:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.684+0000 7faeb67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb80cd5d0 0x7faeb815b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.684+0000 7faeb67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb80cd5d0 0x7faeb815b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49010/0 (socket says 172.21.15.12:49010) 2023-07-08T12:38:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.684+0000 7faeb67fc700 1 -- 172.21.15.12:0/949808610 learned_addr learned my addr 172.21.15.12:0/949808610 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.684+0000 7faeb6ffd700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faeb8070ae0 0x7faeb815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.684+0000 7faeb77fe700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faeb8158a00 0x7faeb81e3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.685+0000 7faeb67fc700 1 -- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faeb8158a00 msgr2=0x7faeb81e3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.685+0000 7faeb67fc700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faeb8158a00 0x7faeb81e3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.685+0000 7faeb67fc700 1 -- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faeb8070ae0 msgr2=0x7faeb815ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.685+0000 7faeb67fc700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faeb8070ae0 0x7faeb815ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.685+0000 7faeb67fc700 1 -- 172.21.15.12:0/949808610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea80094a0 con 0x7faeb80cd5d0 2023-07-08T12:38:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.685+0000 7faeb67fc700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb80cd5d0 0x7faeb815b070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faea800ebe0 tx=0x7faea800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.685+0000 7fae9f7fe700 1 -- 172.21.15.12:0/949808610 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea800c850 con 0x7faeb80cd5d0 2023-07-08T12:38:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.685+0000 7fae9f7fe700 1 -- 172.21.15.12:0/949808610 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea800c9b0 con 0x7faeb80cd5d0 2023-07-08T12:38:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.685+0000 7faebd738700 1 -- 172.21.15.12:0/949808610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb000ab80 con 0x7faeb80cd5d0 2023-07-08T12:38:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.685+0000 7fae9f7fe700 1 -- 172.21.15.12:0/949808610 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea800cd40 con 0x7faeb80cd5d0 2023-07-08T12:38:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.685+0000 7faebd738700 1 -- 172.21.15.12:0/949808610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb80aecf0 con 0x7faeb80cd5d0 2023-07-08T12:38:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.686+0000 7faebd738700 1 -- 172.21.15.12:0/949808610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb80cd440 con 0x7faeb80cd5d0 2023-07-08T12:38:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.686+0000 7fae9f7fe700 1 -- 172.21.15.12:0/949808610 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faea8013470 con 0x7faeb80cd5d0 2023-07-08T12:38:06.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.687+0000 7fae9f7fe700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faea406ace0 0x7faea406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:06.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.687+0000 7fae9f7fe700 1 -- 172.21.15.12:0/949808610 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5580+0+0 (secure 0 0 0) 0x7faea80892c0 con 0x7faeb80cd5d0 2023-07-08T12:38:06.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.687+0000 7faeb6ffd700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faea406ace0 0x7faea406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:06.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.688+0000 7faeb6ffd700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faea406ace0 0x7faea406d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faeac005910 tx=0x7faeac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:06.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.692+0000 7fae9f7fe700 1 -- 172.21.15.12:0/949808610 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faea80545e0 con 0x7faeb80cd5d0 2023-07-08T12:38:06.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:06.889+0000 7faebd738700 1 -- 172.21.15.12:0/949808610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7faeb804ea60 con 0x7faeb80cd5d0 2023-07-08T12:38:07.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.521+0000 7fae9f7fe700 1 -- 172.21.15.12:0/949808610 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v22) v1 ==== 171+0+0 (secure 0 0 0) 0x7faea800cea0 con 0x7faeb80cd5d0 2023-07-08T12:38:07.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.526+0000 7faebd738700 1 -- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faea406ace0 msgr2=0x7faea406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:07.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.526+0000 7faebd738700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faea406ace0 0x7faea406d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faeac005910 tx=0x7faeac009040 comp rx=0 tx=0).stop 2023-07-08T12:38:07.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.526+0000 7faebd738700 1 -- 172.21.15.12:0/949808610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb80cd5d0 msgr2=0x7faeb815b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.526+0000 7faebd738700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb80cd5d0 0x7faeb815b070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faea800ebe0 tx=0x7faea800eef0 comp rx=0 tx=0).stop 2023-07-08T12:38:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.527+0000 7faebd738700 1 -- 172.21.15.12:0/949808610 shutdown_connections 2023-07-08T12:38:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.527+0000 7faebd738700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faea406ace0 0x7faea406d190 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faeac005910 tx=0x7faeac009040 comp rx=0 tx=0).stop 2023-07-08T12:38:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.527+0000 7faebd738700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faeb8158a00 0x7faeb81e3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.527+0000 7faebd738700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faeb8070ae0 0x7faeb815ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.527+0000 7faebd738700 1 --2- 172.21.15.12:0/949808610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb80cd5d0 0x7faeb815b070 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.527+0000 7faebd738700 1 -- 172.21.15.12:0/949808610 >> 172.21.15.12:0/949808610 conn(0x7faeb80cf1d0 msgr2=0x7faeb80cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:07.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.527+0000 7faebd738700 1 -- 172.21.15.12:0/949808610 shutdown_connections 2023-07-08T12:38:07.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.527+0000 7faebd738700 1 -- 172.21.15.12:0/949808610 wait complete. 2023-07-08T12:38:07.536 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' created 2023-07-08T12:38:07.555 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-08T12:38:07.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.685+0000 7fb7903d3700 1 Processor -- start 2023-07-08T12:38:07.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.685+0000 7fb7903d3700 1 -- start start 2023-07-08T12:38:07.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.686+0000 7fb7903d3700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb78815a210 0x7fb7880c9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:07.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.686+0000 7fb7903d3700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7880c9b80 0x7fb7880cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:07.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.686+0000 7fb7903d3700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb788069f30 con 0x7fb7880cc3d0 2023-07-08T12:38:07.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.686+0000 7fb7903d3700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7880cb760 con 0x7fb78815a210 2023-07-08T12:38:07.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.686+0000 7fb7903d3700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7880cc850 con 0x7fb7880c9b80 2023-07-08T12:38:07.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.686+0000 7fb78d96e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7880c9b80 0x7fb7880cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:07.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.686+0000 7fb78e16f700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb78815a210 0x7fb7880c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:07.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.686+0000 7fb78e970700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fb7880cc3d0 0x7fb788070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:45636/0 (socket says 172.21.15.12:45636) 2023-07-08T12:38:07.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.686+0000 7fb78e970700 1 -- 172.21.15.12:0/322725891 learned_addr learned my addr 172.21.15.12:0/322725891 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:07.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.687+0000 7fb78d16d700 1 -- 172.21.15.12:0/322725891 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567643676 0 0) 0x7fb788069f30 con 0x7fb7880cc3d0 2023-07-08T12:38:07.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.687+0000 7fb78d16d700 1 -- 172.21.15.12:0/322725891 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7740035b0 con 0x7fb7880cc3d0 2023-07-08T12:38:07.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.687+0000 7fb78d96e700 1 -- 172.21.15.12:0/322725891 >> v2:172.21.15.160:3300/0 conn(0x7fb78815a210 msgr2=0x7fb7880c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:07.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.687+0000 7fb78d96e700 1 --2- 172.21.15.12:0/322725891 >> v2:172.21.15.160:3300/0 conn(0x7fb78815a210 0x7fb7880c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:07.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.687+0000 7fb78d96e700 1 -- 172.21.15.12:0/322725891 >> v1:172.21.15.160:6789/0 conn(0x7fb7880cc3d0 legacy=0x7fb788070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:07.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.687+0000 7fb78d96e700 1 -- 172.21.15.12:0/322725891 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78806a1d0 con 0x7fb7880c9b80 2023-07-08T12:38:07.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.687+0000 7fb78e16f700 1 --2- 172.21.15.12:0/322725891 >> v2:172.21.15.160:3300/0 conn(0x7fb78815a210 0x7fb7880c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:38:07.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.687+0000 7fb78d96e700 1 --2- 172.21.15.12:0/322725891 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7880c9b80 0x7fb7880cbe90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb784009840 tx=0x7fb784009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6169c1d946c5bfe4 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:07.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.687+0000 7fb78d16d700 1 -- 172.21.15.12:0/322725891 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784011830 con 0x7fb7880c9b80 2023-07-08T12:38:07.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.687+0000 7fb78d16d700 1 -- 172.21.15.12:0/322725891 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb784011990 con 0x7fb7880c9b80 2023-07-08T12:38:07.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.687+0000 7fb78d16d700 1 -- 172.21.15.12:0/322725891 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784011d20 con 0x7fb7880c9b80 2023-07-08T12:38:07.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.688+0000 7fb7903d3700 1 -- 172.21.15.12:0/322725891 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7880c9b80 msgr2=0x7fb7880cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:07.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.688+0000 7fb7903d3700 1 --2- 172.21.15.12:0/322725891 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7880c9b80 0x7fb7880cbe90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb784009840 tx=0x7fb784009b50 comp rx=0 tx=0).stop 2023-07-08T12:38:07.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.688+0000 7fb7903d3700 1 -- 172.21.15.12:0/322725891 shutdown_connections 2023-07-08T12:38:07.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.688+0000 7fb7903d3700 1 --2- 172.21.15.12:0/322725891 >> v2:172.21.15.160:3300/0 conn(0x7fb78815a210 0x7fb7880c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:07.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.688+0000 7fb7903d3700 1 --2- 172.21.15.12:0/322725891 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7880c9b80 0x7fb7880cbe90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:07.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.688+0000 7fb7903d3700 1 -- 172.21.15.12:0/322725891 >> 172.21.15.12:0/322725891 conn(0x7fb7880cf1d0 msgr2=0x7fb7880cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:07.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.688+0000 7fb7903d3700 1 -- 172.21.15.12:0/322725891 shutdown_connections 2023-07-08T12:38:07.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.688+0000 7fb7903d3700 1 -- 172.21.15.12:0/322725891 wait complete. 2023-07-08T12:38:07.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.689+0000 7fb7903d3700 1 Processor -- start 2023-07-08T12:38:07.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.689+0000 7fb7903d3700 1 -- start start 2023-07-08T12:38:07.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb7903d3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7880c9b80 0x7fb78815a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:07.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb7903d3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7880cc3d0 0x7fb788146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:07.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb7903d3700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb78815a210 0x7fb788146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb7903d3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78815c730 con 0x7fb7880c9b80 2023-07-08T12:38:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb7903d3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7881492b0 con 0x7fb7880cc3d0 2023-07-08T12:38:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb7903d3700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb788146dd0 con 0x7fb78815a210 2023-07-08T12:38:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb78e16f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7880c9b80 0x7fb78815a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb78e16f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7880c9b80 0x7fb78815a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49020/0 (socket says 172.21.15.12:49020) 2023-07-08T12:38:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb78e16f700 1 -- 172.21.15.12:0/105841473 learned_addr learned my addr 172.21.15.12:0/105841473 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb78d96e700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7880cc3d0 0x7fb788146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb78e970700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb78815a210 0x7fb788146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:07.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb78e16f700 1 -- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb78815a210 msgr2=0x7fb788146890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:07.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb78e16f700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb78815a210 0x7fb788146890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:07.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb78e16f700 1 -- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7880cc3d0 msgr2=0x7fb788146120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:07.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb78e16f700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7880cc3d0 0x7fb788146120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:07.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.690+0000 7fb78e16f700 1 -- 172.21.15.12:0/105841473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7780094a0 con 0x7fb7880c9b80 2023-07-08T12:38:07.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.691+0000 7fb78e16f700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7880c9b80 0x7fb78815a6c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb77800ec10 tx=0x7fb77800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:07.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.691+0000 7fb77effd700 1 -- 172.21.15.12:0/105841473 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778009760 con 0x7fb7880c9b80 2023-07-08T12:38:07.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.691+0000 7fb77effd700 1 -- 172.21.15.12:0/105841473 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77800c850 con 0x7fb7880c9b80 2023-07-08T12:38:07.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.691+0000 7fb7903d3700 1 -- 172.21.15.12:0/105841473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7840094a0 con 0x7fb7880c9b80 2023-07-08T12:38:07.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.691+0000 7fb77effd700 1 -- 172.21.15.12:0/105841473 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77800cbe0 con 0x7fb7880c9b80 2023-07-08T12:38:07.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.691+0000 7fb7903d3700 1 -- 172.21.15.12:0/105841473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78815c070 con 0x7fb7880c9b80 2023-07-08T12:38:07.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.692+0000 7fb7903d3700 1 -- 172.21.15.12:0/105841473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb788066e80 con 0x7fb7880c9b80 2023-07-08T12:38:07.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.692+0000 7fb77effd700 1 -- 172.21.15.12:0/105841473 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb77800cd40 con 0x7fb7880c9b80 2023-07-08T12:38:07.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.693+0000 7fb77effd700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb76c06ace0 0x7fb76c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:07.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.693+0000 7fb77effd700 1 -- 172.21.15.12:0/105841473 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fb778089350 con 0x7fb7880c9b80 2023-07-08T12:38:07.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.693+0000 7fb78d96e700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb76c06ace0 0x7fb76c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:07.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.694+0000 7fb78d96e700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb76c06ace0 0x7fb76c06d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb784000c00 tx=0x7fb784008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:07.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.698+0000 7fb77effd700 1 -- 172.21.15.12:0/105841473 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb7780544f0 con 0x7fb7880c9b80 2023-07-08T12:38:07.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:07.941+0000 7fb7903d3700 1 -- 172.21.15.12:0/105841473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb78804ea60 con 0x7fb7880c9b80 2023-07-08T12:38:08.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.550+0000 7fb77effd700 1 -- 172.21.15.12:0/105841473 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb778057b10 con 0x7fb7880c9b80 2023-07-08T12:38:08.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.562+0000 7fb7903d3700 1 -- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb76c06ace0 msgr2=0x7fb76c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:08.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.562+0000 7fb7903d3700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb76c06ace0 0x7fb76c06d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb784000c00 tx=0x7fb784008030 comp rx=0 tx=0).stop 2023-07-08T12:38:08.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.562+0000 7fb7903d3700 1 -- 172.21.15.12:0/105841473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7880c9b80 msgr2=0x7fb78815a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:08.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.562+0000 7fb7903d3700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7880c9b80 0x7fb78815a6c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb77800ec10 tx=0x7fb77800ef20 comp rx=0 tx=0).stop 2023-07-08T12:38:08.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.562+0000 7fb7903d3700 1 -- 172.21.15.12:0/105841473 shutdown_connections 2023-07-08T12:38:08.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.562+0000 7fb7903d3700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb76c06ace0 0x7fb76c06d190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:08.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.562+0000 7fb7903d3700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb78815a210 0x7fb788146890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:08.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.562+0000 7fb7903d3700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7880c9b80 0x7fb78815a6c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:08.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.562+0000 7fb7903d3700 1 --2- 172.21.15.12:0/105841473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7880cc3d0 0x7fb788146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:08.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.562+0000 7fb7903d3700 1 -- 172.21.15.12:0/105841473 >> 172.21.15.12:0/105841473 conn(0x7fb7880cf1d0 msgr2=0x7fb788070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:08.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.562+0000 7fb7903d3700 1 -- 172.21.15.12:0/105841473 shutdown_connections 2023-07-08T12:38:08.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.562+0000 7fb7903d3700 1 -- 172.21.15.12:0/105841473 wait complete. 2023-07-08T12:38:08.568 INFO:teuthology.orchestra.run.smithi012.stderr:new fs with metadata pool 3 and data pool 4 2023-07-08T12:38:08.584 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-07-08T12:38:08.585 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-07-08T12:38:08.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.717+0000 7f367a43f700 1 Processor -- start 2023-07-08T12:38:08.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.717+0000 7f367a43f700 1 -- start start 2023-07-08T12:38:08.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.718+0000 7f367a43f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f367415d5d0 0x7f3674069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:08.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.718+0000 7f367a43f700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f367406a470 0x7f36740cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:08.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.718+0000 7f367a43f700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36740715c0 con 0x7f3674159ff0 2023-07-08T12:38:08.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.718+0000 7f367a43f700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36740cdbf0 con 0x7f367406a470 2023-07-08T12:38:08.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.718+0000 7f367a43f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36740725d0 con 0x7f367415d5d0 2023-07-08T12:38:08.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.718+0000 7f3673fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3674159ff0 0x7f36740cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36436/0 (socket says 172.21.15.12:36436) 2023-07-08T12:38:08.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.718+0000 7f3673fff700 1 -- 172.21.15.12:0/3252365691 learned_addr learned my addr 172.21.15.12:0/3252365691 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.718+0000 7f36737fe700 1 --2- 172.21.15.12:0/3252365691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f367415d5d0 0x7f3674069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.718+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097336912 0 0) 0x7f36740715c0 con 0x7f3674159ff0 2023-07-08T12:38:08.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.718+0000 7f36789dc700 1 --2- 172.21.15.12:0/3252365691 >> v2:172.21.15.160:3300/0 conn(0x7f367406a470 0x7f36740cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.718+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f365c0035b0 con 0x7f3674159ff0 2023-07-08T12:38:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1251058849 0 0) 0x7f365c0035b0 con 0x7f3674159ff0 2023-07-08T12:38:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f367415d5d0 msgr2=0x7f3674069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:08.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 --2- 172.21.15.12:0/3252365691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f367415d5d0 0x7f3674069f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 >> v2:172.21.15.160:3300/0 conn(0x7f367406a470 msgr2=0x7f36740cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:08.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 --2- 172.21.15.12:0/3252365691 >> v2:172.21.15.160:3300/0 conn(0x7f367406a470 0x7f36740cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367406a890 con 0x7f3674159ff0 2023-07-08T12:38:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f3668001670 con 0x7f3674159ff0 2023-07-08T12:38:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 >> v1:172.21.15.12:6789/0 conn(0x7f3674159ff0 legacy=0x7f36740cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:08.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 --2- 172.21.15.12:0/3252365691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365c003a00 0x7f365c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:08.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 --2- 172.21.15.12:0/3252365691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f367415d5d0 0x7f365c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:08.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 --2- 172.21.15.12:0/3252365691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f365c004d90 0x7f365c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:08.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36740725d0 con 0x7f365c003a00 2023-07-08T12:38:08.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c00ba10 con 0x7f365c004d90 2023-07-08T12:38:08.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365c00bb50 con 0x7f367415d5d0 2023-07-08T12:38:08.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3673fff700 1 --2- 172.21.15.12:0/3252365691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365c003a00 0x7f365c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:08.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f36737fe700 1 --2- 172.21.15.12:0/3252365691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f367415d5d0 0x7f365c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:08.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f36789dc700 1 --2- 172.21.15.12:0/3252365691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f365c004d90 0x7f365c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:08.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3673fff700 1 -- 172.21.15.12:0/3252365691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f367415d5d0 msgr2=0x7f365c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3673fff700 1 --2- 172.21.15.12:0/3252365691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f367415d5d0 0x7f365c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3673fff700 1 -- 172.21.15.12:0/3252365691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f365c004d90 msgr2=0x7f365c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3673fff700 1 --2- 172.21.15.12:0/3252365691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f365c004d90 0x7f365c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3673fff700 1 -- 172.21.15.12:0/3252365691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0035b0 con 0x7f365c003a00 2023-07-08T12:38:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3673fff700 1 --2- 172.21.15.12:0/3252365691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365c003a00 0x7f365c003e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f366800c2f0 tx=0x7f365c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1242aabbe4a3e2ba server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3668007c40 con 0x7f365c003a00 2023-07-08T12:38:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366800ce30 con 0x7f365c003a00 2023-07-08T12:38:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.719+0000 7f3672ffd700 1 -- 172.21.15.12:0/3252365691 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3668005ba0 con 0x7f365c003a00 2023-07-08T12:38:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.720+0000 7f367a43f700 1 -- 172.21.15.12:0/3252365691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365c003a00 msgr2=0x7f365c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.720+0000 7f367a43f700 1 --2- 172.21.15.12:0/3252365691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365c003a00 0x7f365c003e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f366800c2f0 tx=0x7f365c005880 comp rx=0 tx=0).stop 2023-07-08T12:38:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.720+0000 7f36789dc700 1 -- 172.21.15.12:0/3252365691 reap_dead start 2023-07-08T12:38:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.720+0000 7f367a43f700 1 -- 172.21.15.12:0/3252365691 shutdown_connections 2023-07-08T12:38:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.720+0000 7f367a43f700 1 -- 172.21.15.12:0/3252365691 >> 172.21.15.12:0/3252365691 conn(0x7f367406fd80 msgr2=0x7f36740b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.720+0000 7f367a43f700 1 -- 172.21.15.12:0/3252365691 shutdown_connections 2023-07-08T12:38:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.720+0000 7f367a43f700 1 -- 172.21.15.12:0/3252365691 wait complete. 2023-07-08T12:38:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.721+0000 7f367a43f700 1 Processor -- start 2023-07-08T12:38:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.721+0000 7f367a43f700 1 -- start start 2023-07-08T12:38:08.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.722+0000 7f367a43f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f365c003a00 0x7f36740c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:08.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.722+0000 7f367a43f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f367414dfb0 0x7f36741e7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:08.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.722+0000 7f367a43f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f367414d9f0 0x7f36741e8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:08.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.722+0000 7f367a43f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36740af5e0 con 0x7f367414d9f0 2023-07-08T12:38:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.722+0000 7f367a43f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36740c43e0 con 0x7f367414dfb0 2023-07-08T12:38:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.722+0000 7f367a43f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36741e8a60 con 0x7f365c003a00 2023-07-08T12:38:08.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.722+0000 7f36789dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f367414d9f0 0x7f36741e8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:08.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.722+0000 7f36789dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f367414d9f0 0x7f36741e8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49054/0 (socket says 172.21.15.12:49054) 2023-07-08T12:38:08.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.722+0000 7f36789dc700 1 -- 172.21.15.12:0/1114157762 learned_addr learned my addr 172.21.15.12:0/1114157762 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:08.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.722+0000 7f3673fff700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f365c003a00 0x7f36740c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:08.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.722+0000 7f36737fe700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f367414dfb0 0x7f36741e7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:08.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.723+0000 7f36789dc700 1 -- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f365c003a00 msgr2=0x7f36740c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:08.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.723+0000 7f36789dc700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f365c003a00 0x7f36740c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:08.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.723+0000 7f36789dc700 1 -- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f367414dfb0 msgr2=0x7f36741e7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:08.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.723+0000 7f36789dc700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f367414dfb0 0x7f36741e7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:08.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.723+0000 7f36789dc700 1 -- 172.21.15.12:0/1114157762 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36740c3e20 con 0x7f367414d9f0 2023-07-08T12:38:08.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.723+0000 7f36789dc700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f367414d9f0 0x7f36741e8520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f366c011bf0 tx=0x7f366c00f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:08.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.723+0000 7f3670ff9700 1 -- 172.21.15.12:0/1114157762 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c00e040 con 0x7f367414d9f0 2023-07-08T12:38:08.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.723+0000 7f3670ff9700 1 -- 172.21.15.12:0/1114157762 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c00f040 con 0x7f367414d9f0 2023-07-08T12:38:08.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.723+0000 7f367a43f700 1 -- 172.21.15.12:0/1114157762 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36740c4100 con 0x7f367414d9f0 2023-07-08T12:38:08.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.723+0000 7f3670ff9700 1 -- 172.21.15.12:0/1114157762 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c00fdf0 con 0x7f367414d9f0 2023-07-08T12:38:08.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.723+0000 7f367a43f700 1 -- 172.21.15.12:0/1114157762 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36740c61e0 con 0x7f367414d9f0 2023-07-08T12:38:08.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.724+0000 7f367a43f700 1 -- 172.21.15.12:0/1114157762 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36740af750 con 0x7f367414d9f0 2023-07-08T12:38:08.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.724+0000 7f3670ff9700 1 -- 172.21.15.12:0/1114157762 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f366c022460 con 0x7f367414d9f0 2023-07-08T12:38:08.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.725+0000 7f3670ff9700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f366006ace0 0x7f366006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.725+0000 7f3670ff9700 1 -- 172.21.15.12:0/1114157762 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f366c089ac0 con 0x7f367414d9f0 2023-07-08T12:38:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.725+0000 7f3673fff700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f366006ace0 0x7f366006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:08.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.725+0000 7f3673fff700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f366006ace0 0x7f366006d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3668008fd0 tx=0x7f366800a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:08.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.729+0000 7f3670ff9700 1 -- 172.21.15.12:0/1114157762 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f36740af750 con 0x7f367414d9f0 2023-07-08T12:38:08.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:08.938+0000 7f367a43f700 1 -- 172.21.15.12:0/1114157762 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f36740af720 con 0x7f367414d9f0 2023-07-08T12:38:09.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.551+0000 7f3670ff9700 1 -- 172.21.15.12:0/1114157762 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v24) v1 ==== 157+0+0 (secure 0 0 0) 0x7f366c054b70 con 0x7f367414d9f0 2023-07-08T12:38:09.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.554+0000 7f367a43f700 1 -- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f366006ace0 msgr2=0x7f366006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:09.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.554+0000 7f367a43f700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f366006ace0 0x7f366006d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3668008fd0 tx=0x7f366800a000 comp rx=0 tx=0).stop 2023-07-08T12:38:09.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.554+0000 7f367a43f700 1 -- 172.21.15.12:0/1114157762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f367414d9f0 msgr2=0x7f36741e8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:09.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.554+0000 7f367a43f700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f367414d9f0 0x7f36741e8520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f366c011bf0 tx=0x7f366c00f1b0 comp rx=0 tx=0).stop 2023-07-08T12:38:09.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.554+0000 7f367a43f700 1 -- 172.21.15.12:0/1114157762 shutdown_connections 2023-07-08T12:38:09.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.554+0000 7f367a43f700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f366006ace0 0x7f366006d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:09.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.554+0000 7f367a43f700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f367414d9f0 0x7f36741e8520 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:09.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.554+0000 7f367a43f700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f365c003a00 0x7f36740c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:09.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.554+0000 7f367a43f700 1 --2- 172.21.15.12:0/1114157762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f367414dfb0 0x7f36741e7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:09.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.554+0000 7f367a43f700 1 -- 172.21.15.12:0/1114157762 >> 172.21.15.12:0/1114157762 conn(0x7f367406fd80 msgr2=0x7f367415dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:09.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.554+0000 7f367a43f700 1 -- 172.21.15.12:0/1114157762 shutdown_connections 2023-07-08T12:38:09.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.554+0000 7f367a43f700 1 -- 172.21.15.12:0/1114157762 wait complete. 2023-07-08T12:38:09.571 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-07-08T12:38:09.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.712+0000 7fce030a3700 1 Processor -- start 2023-07-08T12:38:09.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.712+0000 7fce030a3700 1 -- start start 2023-07-08T12:38:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.712+0000 7fce030a3700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcdfc0cb3d0 0x7fcdfc070d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.712+0000 7fce030a3700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdfc0abde0 con 0x7fcdfc0712c0 2023-07-08T12:38:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.712+0000 7fce030a3700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdfc071720 con 0x7fcdfc0ada00 2023-07-08T12:38:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.712+0000 7fce030a3700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdfc06a640 con 0x7fcdfc0cb3d0 2023-07-08T12:38:09.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.713+0000 7fcdfbfff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcdfc0cb3d0 0x7fcdfc070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:09.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.713+0000 7fce01640700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fcdfc0712c0 0x7fcdfc0c9e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36460/0 (socket says 172.21.15.12:36460) 2023-07-08T12:38:09.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.713+0000 7fce01640700 1 -- 172.21.15.12:0/2954932118 learned_addr learned my addr 172.21.15.12:0/2954932118 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:09.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.713+0000 7fcdfbfff700 1 -- 172.21.15.12:0/2954932118 >> v1:172.21.15.160:6789/0 conn(0x7fcdfc0ada00 legacy=0x7fcdfc069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:09.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.713+0000 7fcdfbfff700 1 -- 172.21.15.12:0/2954932118 >> v1:172.21.15.12:6789/0 conn(0x7fcdfc0712c0 legacy=0x7fcdfc0c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:09.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.713+0000 7fcdfb7fe700 1 -- 172.21.15.12:0/2954932118 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7315231 0 0) 0x7fcdfc0abde0 con 0x7fcdfc0712c0 2023-07-08T12:38:09.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.713+0000 7fcdfbfff700 1 -- 172.21.15.12:0/2954932118 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc0c9700 con 0x7fcdfc0cb3d0 2023-07-08T12:38:09.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.713+0000 7fcdfbfff700 1 --2- 172.21.15.12:0/2954932118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdfc0cb3d0 0x7fcdfc070d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009840 tx=0x7fcde8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbe95e7b5d02dec server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:09.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.713+0000 7fcdfb7fe700 1 -- 172.21.15.12:0/2954932118 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8011830 con 0x7fcdfc0cb3d0 2023-07-08T12:38:09.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.713+0000 7fcdfb7fe700 1 -- 172.21.15.12:0/2954932118 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde8011990 con 0x7fcdfc0cb3d0 2023-07-08T12:38:09.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.713+0000 7fcdfb7fe700 1 -- 172.21.15.12:0/2954932118 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8011d20 con 0x7fcdfc0cb3d0 2023-07-08T12:38:09.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.714+0000 7fce030a3700 1 -- 172.21.15.12:0/2954932118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdfc0cb3d0 msgr2=0x7fcdfc070d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:09.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.714+0000 7fce030a3700 1 --2- 172.21.15.12:0/2954932118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdfc0cb3d0 0x7fcdfc070d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009840 tx=0x7fcde8009b50 comp rx=0 tx=0).stop 2023-07-08T12:38:09.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.714+0000 7fce030a3700 1 -- 172.21.15.12:0/2954932118 shutdown_connections 2023-07-08T12:38:09.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.714+0000 7fce030a3700 1 --2- 172.21.15.12:0/2954932118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdfc0cb3d0 0x7fcdfc070d80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:09.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.714+0000 7fce030a3700 1 -- 172.21.15.12:0/2954932118 >> 172.21.15.12:0/2954932118 conn(0x7fcdfc0cf0b0 msgr2=0x7fcdfc0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:09.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.714+0000 7fce030a3700 1 -- 172.21.15.12:0/2954932118 shutdown_connections 2023-07-08T12:38:09.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.714+0000 7fce030a3700 1 -- 172.21.15.12:0/2954932118 wait complete. 2023-07-08T12:38:09.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.715+0000 7fce030a3700 1 Processor -- start 2023-07-08T12:38:09.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.715+0000 7fce030a3700 1 -- start start 2023-07-08T12:38:09.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.716+0000 7fce030a3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdfc0712c0 0x7fcdfc152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:09.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.716+0000 7fce030a3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdfc0ada00 0x7fcdfc145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:09.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.716+0000 7fce030a3700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcdfc0cb3d0 0x7fcdfc14e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:09.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.716+0000 7fce030a3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc1510b0 con 0x7fcdfc0ada00 2023-07-08T12:38:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.716+0000 7fce030a3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc1512c0 con 0x7fcdfc0712c0 2023-07-08T12:38:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.716+0000 7fce030a3700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc1515a0 con 0x7fcdfc0cb3d0 2023-07-08T12:38:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.716+0000 7fcdfbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdfc0ada00 0x7fcdfc145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.716+0000 7fcdfbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdfc0ada00 0x7fcdfc145610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49068/0 (socket says 172.21.15.12:49068) 2023-07-08T12:38:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.716+0000 7fcdfbfff700 1 -- 172.21.15.12:0/3541304315 learned_addr learned my addr 172.21.15.12:0/3541304315 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fce00e3f700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdfc0712c0 0x7fcdfc152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fce01640700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcdfc0cb3d0 0x7fcdfc14e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fcdfbfff700 1 -- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcdfc0cb3d0 msgr2=0x7fcdfc14e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fcdfbfff700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcdfc0cb3d0 0x7fcdfc14e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fcdfbfff700 1 -- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdfc0712c0 msgr2=0x7fcdfc152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fcdfbfff700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdfc0712c0 0x7fcdfc152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fcdfbfff700 1 -- 172.21.15.12:0/3541304315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde80094a0 con 0x7fcdfc0ada00 2023-07-08T12:38:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fce00e3f700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdfc0712c0 0x7fcdfc152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fcdfbfff700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdfc0ada00 0x7fcdfc145610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcde8014040 tx=0x7fcde8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fcdf97fa700 1 -- 172.21.15.12:0/3541304315 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde801a900 con 0x7fcdfc0ada00 2023-07-08T12:38:09.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fcdf97fa700 1 -- 172.21.15.12:0/3541304315 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde801aa60 con 0x7fcdfc0ada00 2023-07-08T12:38:09.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fce030a3700 1 -- 172.21.15.12:0/3541304315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdfc151880 con 0x7fcdfc0ada00 2023-07-08T12:38:09.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fcdf97fa700 1 -- 172.21.15.12:0/3541304315 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde801add0 con 0x7fcdfc0ada00 2023-07-08T12:38:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.717+0000 7fce030a3700 1 -- 172.21.15.12:0/3541304315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc151e40 con 0x7fcdfc0ada00 2023-07-08T12:38:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.718+0000 7fce030a3700 1 -- 172.21.15.12:0/3541304315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdfc04ea60 con 0x7fcdfc0ada00 2023-07-08T12:38:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.718+0000 7fcdf97fa700 1 -- 172.21.15.12:0/3541304315 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcde8022070 con 0x7fcdfc0ada00 2023-07-08T12:38:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.719+0000 7fcdf97fa700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcde006ace0 0x7fcde006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.719+0000 7fcdf97fa700 1 -- 172.21.15.12:0/3541304315 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fcde808a7f0 con 0x7fcdfc0ada00 2023-07-08T12:38:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.719+0000 7fce00e3f700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcde006ace0 0x7fcde006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.720+0000 7fce00e3f700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcde006ace0 0x7fcde006d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000f60 tx=0x7fcdf0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.723+0000 7fcdf97fa700 1 -- 172.21.15.12:0/3541304315 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcde8055970 con 0x7fcdfc0ada00 2023-07-08T12:38:09.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:09.921+0000 7fce030a3700 1 -- 172.21.15.12:0/3541304315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcdfc06b850 con 0x7fcdfc0ada00 2023-07-08T12:38:11.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.562+0000 7fcdf97fa700 1 -- 172.21.15.12:0/3541304315 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v26) v1 ==== 228+0+0 (secure 0 0 0) 0x7fcde8058f90 con 0x7fcdfc0ada00 2023-07-08T12:38:11.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.569+0000 7fce030a3700 1 -- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcde006ace0 msgr2=0x7fcde006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:11.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.569+0000 7fce030a3700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcde006ace0 0x7fcde006d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000f60 tx=0x7fcdf0009040 comp rx=0 tx=0).stop 2023-07-08T12:38:11.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.569+0000 7fce030a3700 1 -- 172.21.15.12:0/3541304315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdfc0ada00 msgr2=0x7fcdfc145610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:11.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.569+0000 7fce030a3700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdfc0ada00 0x7fcdfc145610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcde8014040 tx=0x7fcde8004750 comp rx=0 tx=0).stop 2023-07-08T12:38:11.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.571+0000 7fce030a3700 1 -- 172.21.15.12:0/3541304315 shutdown_connections 2023-07-08T12:38:11.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.571+0000 7fce030a3700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcde006ace0 0x7fcde006d190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:11.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.571+0000 7fce030a3700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcdfc0cb3d0 0x7fcdfc14e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:11.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.571+0000 7fce030a3700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdfc0712c0 0x7fcdfc152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:11.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.571+0000 7fce030a3700 1 --2- 172.21.15.12:0/3541304315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdfc0ada00 0x7fcdfc145610 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:11.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.571+0000 7fce030a3700 1 -- 172.21.15.12:0/3541304315 >> 172.21.15.12:0/3541304315 conn(0x7fcdfc0cf0b0 msgr2=0x7fcdfc069f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:11.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.572+0000 7fce030a3700 1 -- 172.21.15.12:0/3541304315 shutdown_connections 2023-07-08T12:38:11.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.573+0000 7fce030a3700 1 -- 172.21.15.12:0/3541304315 wait complete. 2023-07-08T12:38:11.577 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data_ec' created 2023-07-08T12:38:11.591 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-07-08T12:38:11.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.724+0000 7fd99e461700 1 Processor -- start 2023-07-08T12:38:11.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.724+0000 7fd99e461700 1 -- start start 2023-07-08T12:38:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd99e461700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd9980cdac0 0x7fd998071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd99e461700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9980b1890 con 0x7fd9980b1d20 2023-07-08T12:38:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd99e461700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9980b2170 con 0x7fd9981590c0 2023-07-08T12:38:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd99e461700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99815a4a0 con 0x7fd9980cdac0 2023-07-08T12:38:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd9977fe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd9980b1d20 0x7fd9980cd3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36472/0 (socket says 172.21.15.12:36472) 2023-07-08T12:38:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd9977fe700 1 -- 172.21.15.12:0/1356930669 learned_addr learned my addr 172.21.15.12:0/1356930669 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd99c9fe700 1 --2- 172.21.15.12:0/1356930669 >> v2:172.21.15.160:3300/0 conn(0x7fd9980cdac0 0x7fd998071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd996ffd700 1 -- 172.21.15.12:0/1356930669 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427955052 0 0) 0x7fd9980b1890 con 0x7fd9980b1d20 2023-07-08T12:38:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd996ffd700 1 -- 172.21.15.12:0/1356930669 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9800035b0 con 0x7fd9980b1d20 2023-07-08T12:38:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd996ffd700 1 -- 172.21.15.12:0/1356930669 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713842467 0 0) 0x7fd9980b2170 con 0x7fd9981590c0 2023-07-08T12:38:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd996ffd700 1 -- 172.21.15.12:0/1356930669 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9980b1890 con 0x7fd9981590c0 2023-07-08T12:38:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd996ffd700 1 -- 172.21.15.12:0/1356930669 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2926713804 0 0) 0x7fd9800035b0 con 0x7fd9980b1d20 2023-07-08T12:38:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd99c9fe700 1 -- 172.21.15.12:0/1356930669 >> v1:172.21.15.160:6789/0 conn(0x7fd9981590c0 legacy=0x7fd99815cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd99c9fe700 1 -- 172.21.15.12:0/1356930669 >> v1:172.21.15.12:6789/0 conn(0x7fd9980b1d20 legacy=0x7fd9980cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.725+0000 7fd99c9fe700 1 -- 172.21.15.12:0/1356930669 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9980b1260 con 0x7fd9980cdac0 2023-07-08T12:38:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.726+0000 7fd99c9fe700 1 --2- 172.21.15.12:0/1356930669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9980cdac0 0x7fd998071450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd990009d90 tx=0x7fd99000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307c1da5870cf75f server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.726+0000 7fd996ffd700 1 -- 172.21.15.12:0/1356930669 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99000e070 con 0x7fd9980cdac0 2023-07-08T12:38:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.726+0000 7fd996ffd700 1 -- 172.21.15.12:0/1356930669 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd990003a20 con 0x7fd9980cdac0 2023-07-08T12:38:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.726+0000 7fd996ffd700 1 -- 172.21.15.12:0/1356930669 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c001670 con 0x7fd9980cdac0 2023-07-08T12:38:11.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.726+0000 7fd99e461700 1 -- 172.21.15.12:0/1356930669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9980cdac0 msgr2=0x7fd998071450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:11.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.726+0000 7fd99e461700 1 --2- 172.21.15.12:0/1356930669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9980cdac0 0x7fd998071450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd990009d90 tx=0x7fd99000b330 comp rx=0 tx=0).stop 2023-07-08T12:38:11.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.726+0000 7fd99e461700 1 -- 172.21.15.12:0/1356930669 shutdown_connections 2023-07-08T12:38:11.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.726+0000 7fd99e461700 1 --2- 172.21.15.12:0/1356930669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9980cdac0 0x7fd998071450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:11.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.726+0000 7fd99e461700 1 -- 172.21.15.12:0/1356930669 >> 172.21.15.12:0/1356930669 conn(0x7fd99806d450 msgr2=0x7fd99806d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:11.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.726+0000 7fd99e461700 1 -- 172.21.15.12:0/1356930669 shutdown_connections 2023-07-08T12:38:11.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.726+0000 7fd99e461700 1 -- 172.21.15.12:0/1356930669 wait complete. 2023-07-08T12:38:11.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.728+0000 7fd99e461700 1 Processor -- start 2023-07-08T12:38:11.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.728+0000 7fd99e461700 1 -- start start 2023-07-08T12:38:11.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.728+0000 7fd99e461700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9980b1d20 0x7fd99806a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:11.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.728+0000 7fd99e461700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9980cdac0 0x7fd998149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:11.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.728+0000 7fd99e461700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9981590c0 0x7fd9980ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:11.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.728+0000 7fd99e461700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9981f64d0 con 0x7fd9981590c0 2023-07-08T12:38:11.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.728+0000 7fd99e461700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99806ae50 con 0x7fd9980b1d20 2023-07-08T12:38:11.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.728+0000 7fd99e461700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9981f5950 con 0x7fd9980cdac0 2023-07-08T12:38:11.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd99c9fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9981590c0 0x7fd9980ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:11.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd99c9fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9981590c0 0x7fd9980ca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49082/0 (socket says 172.21.15.12:49082) 2023-07-08T12:38:11.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd99c9fe700 1 -- 172.21.15.12:0/3844025296 learned_addr learned my addr 172.21.15.12:0/3844025296 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:11.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd997fff700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9980b1d20 0x7fd99806a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:11.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd9977fe700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9980cdac0 0x7fd998149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:11.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd99c9fe700 1 -- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9980cdac0 msgr2=0x7fd998149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:11.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd99c9fe700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9980cdac0 0x7fd998149bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:11.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd99c9fe700 1 -- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9980b1d20 msgr2=0x7fd99806a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:11.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd99c9fe700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9980b1d20 0x7fd99806a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:11.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd99c9fe700 1 -- 172.21.15.12:0/3844025296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd990009a70 con 0x7fd9981590c0 2023-07-08T12:38:11.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd99c9fe700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9981590c0 0x7fd9980ca8e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd99000b370 tx=0x7fd990004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:11.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd994ff9700 1 -- 172.21.15.12:0/3844025296 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9900045c0 con 0x7fd9981590c0 2023-07-08T12:38:11.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd994ff9700 1 -- 172.21.15.12:0/3844025296 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd990004720 con 0x7fd9981590c0 2023-07-08T12:38:11.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd994ff9700 1 -- 172.21.15.12:0/3844025296 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99001d470 con 0x7fd9981590c0 2023-07-08T12:38:11.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd99e461700 1 -- 172.21.15.12:0/3844025296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9981f5c30 con 0x7fd9981590c0 2023-07-08T12:38:11.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.729+0000 7fd99e461700 1 -- 172.21.15.12:0/3844025296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9981f61f0 con 0x7fd9981590c0 2023-07-08T12:38:11.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.730+0000 7fd99e461700 1 -- 172.21.15.12:0/3844025296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd998074180 con 0x7fd9981590c0 2023-07-08T12:38:11.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.731+0000 7fd994ff9700 1 -- 172.21.15.12:0/3844025296 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd99001d5d0 con 0x7fd9981590c0 2023-07-08T12:38:11.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.731+0000 7fd994ff9700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd98406ace0 0x7fd98406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:11.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.731+0000 7fd994ff9700 1 -- 172.21.15.12:0/3844025296 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6781+0+0 (secure 0 0 0) 0x7fd99008ad70 con 0x7fd9981590c0 2023-07-08T12:38:11.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.732+0000 7fd997fff700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd98406ace0 0x7fd98406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:11.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.732+0000 7fd997fff700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd98406ace0 0x7fd98406d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd988007fd0 tx=0x7fd988009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:11.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.736+0000 7fd994ff9700 1 -- 172.21.15.12:0/3844025296 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd990055ad0 con 0x7fd9981590c0 2023-07-08T12:38:11.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:11.936+0000 7fd99e461700 1 -- 172.21.15.12:0/3844025296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fd9980b5760 con 0x7fd9981590c0 2023-07-08T12:38:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.566+0000 7fd994ff9700 1 -- 172.21.15.12:0/3844025296 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd9900590f0 con 0x7fd9981590c0 2023-07-08T12:38:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.579+0000 7fd99e461700 1 -- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd98406ace0 msgr2=0x7fd98406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.579+0000 7fd99e461700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd98406ace0 0x7fd98406d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd988007fd0 tx=0x7fd988009040 comp rx=0 tx=0).stop 2023-07-08T12:38:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.579+0000 7fd99e461700 1 -- 172.21.15.12:0/3844025296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9981590c0 msgr2=0x7fd9980ca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.579+0000 7fd99e461700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9981590c0 0x7fd9980ca8e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd99000b370 tx=0x7fd990004ad0 comp rx=0 tx=0).stop 2023-07-08T12:38:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.579+0000 7fd99e461700 1 -- 172.21.15.12:0/3844025296 shutdown_connections 2023-07-08T12:38:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.579+0000 7fd99e461700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd98406ace0 0x7fd98406d190 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd988007fd0 tx=0x7fd988009040 comp rx=0 tx=0).stop 2023-07-08T12:38:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.579+0000 7fd99e461700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9981590c0 0x7fd9980ca8e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.579+0000 7fd99e461700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9980b1d20 0x7fd99806a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.579+0000 7fd99e461700 1 --2- 172.21.15.12:0/3844025296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9980cdac0 0x7fd998149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:12.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.579+0000 7fd99e461700 1 -- 172.21.15.12:0/3844025296 >> 172.21.15.12:0/3844025296 conn(0x7fd99806d450 msgr2=0x7fd9980cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:12.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.581+0000 7fd99e461700 1 -- 172.21.15.12:0/3844025296 shutdown_connections 2023-07-08T12:38:12.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.581+0000 7fd99e461700 1 -- 172.21.15.12:0/3844025296 wait complete. 2023-07-08T12:38:12.596 INFO:teuthology.orchestra.run.smithi012.stderr:set pool 5 allow_ec_overwrites to true 2023-07-08T12:38:12.607 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-07-08T12:38:12.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.736+0000 7f6440b65700 1 Processor -- start 2023-07-08T12:38:12.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.736+0000 7f6440b65700 1 -- start start 2023-07-08T12:38:12.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f6440b65700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f643c159a70 0x7f643c0cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:12.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f6440b65700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0cd5a0 0x7f643c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:12.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f6440b65700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c0cda00 con 0x7f643c1596a0 2023-07-08T12:38:12.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f6440b65700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643c15c420 con 0x7f643c159a70 2023-07-08T12:38:12.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f6440b65700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c15c6c0 con 0x7f643c0cd5a0 2023-07-08T12:38:12.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f643a59c700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f643c1596a0 0x7f643c070bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36484/0 (socket says 172.21.15.12:36484) 2023-07-08T12:38:12.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f643a59c700 1 -- 172.21.15.12:0/1505320684 learned_addr learned my addr 172.21.15.12:0/1505320684 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:12.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f643ad9d700 1 --2- 172.21.15.12:0/1505320684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0cd5a0 0x7f643c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:12.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f6439d9b700 1 --2- 172.21.15.12:0/1505320684 >> v2:172.21.15.160:3300/0 conn(0x7f643c159a70 0x7f643c0cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:12.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f643959a700 1 -- 172.21.15.12:0/1505320684 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891257383 0 0) 0x7f643c0cda00 con 0x7f643c1596a0 2023-07-08T12:38:12.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f643959a700 1 -- 172.21.15.12:0/1505320684 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64240035b0 con 0x7f643c1596a0 2023-07-08T12:38:12.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f643ad9d700 1 -- 172.21.15.12:0/1505320684 >> v2:172.21.15.160:3300/0 conn(0x7f643c159a70 msgr2=0x7f643c0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:12.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f643ad9d700 1 --2- 172.21.15.12:0/1505320684 >> v2:172.21.15.160:3300/0 conn(0x7f643c159a70 0x7f643c0cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:12.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f643ad9d700 1 -- 172.21.15.12:0/1505320684 >> v1:172.21.15.12:6789/0 conn(0x7f643c1596a0 legacy=0x7f643c070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:12.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.737+0000 7f643ad9d700 1 -- 172.21.15.12:0/1505320684 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c15c9a0 con 0x7f643c0cd5a0 2023-07-08T12:38:12.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.738+0000 7f643ad9d700 1 --2- 172.21.15.12:0/1505320684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0cd5a0 0x7f643c069f30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6434009d90 tx=0x7f643400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d2851dc2f8bef36 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:12.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.738+0000 7f643959a700 1 -- 172.21.15.12:0/1505320684 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400e070 con 0x7f643c0cd5a0 2023-07-08T12:38:12.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.738+0000 7f643959a700 1 -- 172.21.15.12:0/1505320684 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643400bb20 con 0x7f643c0cd5a0 2023-07-08T12:38:12.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.738+0000 7f643959a700 1 -- 172.21.15.12:0/1505320684 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434012430 con 0x7f643c0cd5a0 2023-07-08T12:38:12.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.738+0000 7f6440b65700 1 -- 172.21.15.12:0/1505320684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0cd5a0 msgr2=0x7f643c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:12.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.738+0000 7f6440b65700 1 --2- 172.21.15.12:0/1505320684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0cd5a0 0x7f643c069f30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6434009d90 tx=0x7f643400b330 comp rx=0 tx=0).stop 2023-07-08T12:38:12.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.738+0000 7f6440b65700 1 -- 172.21.15.12:0/1505320684 shutdown_connections 2023-07-08T12:38:12.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.738+0000 7f6440b65700 1 --2- 172.21.15.12:0/1505320684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0cd5a0 0x7f643c069f30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:12.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.738+0000 7f6440b65700 1 --2- 172.21.15.12:0/1505320684 >> v2:172.21.15.160:3300/0 conn(0x7f643c159a70 0x7f643c0cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:12.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.738+0000 7f6440b65700 1 -- 172.21.15.12:0/1505320684 >> 172.21.15.12:0/1505320684 conn(0x7f643c0cba70 msgr2=0x7f643c0cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:12.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.739+0000 7f6440b65700 1 -- 172.21.15.12:0/1505320684 shutdown_connections 2023-07-08T12:38:12.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.739+0000 7f6440b65700 1 -- 172.21.15.12:0/1505320684 wait complete. 2023-07-08T12:38:12.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.740+0000 7f6440b65700 1 Processor -- start 2023-07-08T12:38:12.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.740+0000 7f6440b65700 1 -- start start 2023-07-08T12:38:12.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f6440b65700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0cd5a0 0x7f643c0cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:12.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f6440b65700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f643c1596a0 0x7f643c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:12.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f6440b65700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f643c159a70 0x7f643c140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:12.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f6440b65700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0cac60 con 0x7f643c159a70 2023-07-08T12:38:12.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f6440b65700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0ca050 con 0x7f643c1596a0 2023-07-08T12:38:12.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f6440b65700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0ca330 con 0x7f643c0cd5a0 2023-07-08T12:38:12.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f643ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f643c159a70 0x7f643c140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:12.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f643ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f643c159a70 0x7f643c140af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49098/0 (socket says 172.21.15.12:49098) 2023-07-08T12:38:12.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f643ad9d700 1 -- 172.21.15.12:0/2809129543 learned_addr learned my addr 172.21.15.12:0/2809129543 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:12.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f6439d9b700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f643c1596a0 0x7f643c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:12.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f643a59c700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0cd5a0 0x7f643c0cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:12.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f643ad9d700 1 -- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0cd5a0 msgr2=0x7f643c0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:12.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f643ad9d700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0cd5a0 0x7f643c0cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:12.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f643ad9d700 1 -- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f643c1596a0 msgr2=0x7f643c0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:12.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f643ad9d700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f643c1596a0 0x7f643c0c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:12.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f643ad9d700 1 -- 172.21.15.12:0/2809129543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6434009a70 con 0x7f643c159a70 2023-07-08T12:38:12.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f643ad9d700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f643c159a70 0x7f643c140af0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6434014040 tx=0x7f6434004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:12.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f642affd700 1 -- 172.21.15.12:0/2809129543 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64340129f0 con 0x7f643c159a70 2023-07-08T12:38:12.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f642affd700 1 -- 172.21.15.12:0/2809129543 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6434020070 con 0x7f643c159a70 2023-07-08T12:38:12.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f642affd700 1 -- 172.21.15.12:0/2809129543 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434012db0 con 0x7f643c159a70 2023-07-08T12:38:12.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f6440b65700 1 -- 172.21.15.12:0/2809129543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643c0ca610 con 0x7f643c159a70 2023-07-08T12:38:12.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.741+0000 7f6440b65700 1 -- 172.21.15.12:0/2809129543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c143ec0 con 0x7f643c159a70 2023-07-08T12:38:12.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.742+0000 7f6440b65700 1 -- 172.21.15.12:0/2809129543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643c0cae10 con 0x7f643c159a70 2023-07-08T12:38:12.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.743+0000 7f642affd700 1 -- 172.21.15.12:0/2809129543 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6434004040 con 0x7f643c159a70 2023-07-08T12:38:12.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.743+0000 7f642affd700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f641c06ace0 0x7f641c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:12.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.743+0000 7f642affd700 1 -- 172.21.15.12:0/2809129543 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f6434022070 con 0x7f643c159a70 2023-07-08T12:38:12.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.743+0000 7f643a59c700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f641c06ace0 0x7f641c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:12.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.744+0000 7f643a59c700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f641c06ace0 0x7f641c06d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f642c0011e0 tx=0x7f642c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:12.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.748+0000 7f642affd700 1 -- 172.21.15.12:0/2809129543 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6434056250 con 0x7f643c159a70 2023-07-08T12:38:12.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:12.992+0000 7f6440b65700 1 -- 172.21.15.12:0/2809129543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f643c04ea60 con 0x7f643c159a70 2023-07-08T12:38:13.514 INFO:tasks.ceph.mds.a.smithi012.stderr:2023-07-08T12:38:13.512+0000 7f2bd8677700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-08T12:38:13.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.608+0000 7f642affd700 1 -- 172.21.15.12:0/2809129543 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f6434059870 con 0x7f643c159a70 2023-07-08T12:38:13.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.611+0000 7f6440b65700 1 -- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f641c06ace0 msgr2=0x7f641c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:13.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.611+0000 7f6440b65700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f641c06ace0 0x7f641c06d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f642c0011e0 tx=0x7f642c009040 comp rx=0 tx=0).stop 2023-07-08T12:38:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.611+0000 7f6440b65700 1 -- 172.21.15.12:0/2809129543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f643c159a70 msgr2=0x7f643c140af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.611+0000 7f6440b65700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f643c159a70 0x7f643c140af0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6434014040 tx=0x7f6434004ad0 comp rx=0 tx=0).stop 2023-07-08T12:38:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.611+0000 7f6440b65700 1 -- 172.21.15.12:0/2809129543 shutdown_connections 2023-07-08T12:38:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.611+0000 7f6440b65700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f641c06ace0 0x7f641c06d190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.611+0000 7f6440b65700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f643c159a70 0x7f643c140af0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.611+0000 7f6440b65700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0cd5a0 0x7f643c0cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.611+0000 7f6440b65700 1 --2- 172.21.15.12:0/2809129543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f643c1596a0 0x7f643c0c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:13.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.611+0000 7f6440b65700 1 -- 172.21.15.12:0/2809129543 >> 172.21.15.12:0/2809129543 conn(0x7f643c0cba70 msgr2=0x7f643c0cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:13.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.611+0000 7f6440b65700 1 -- 172.21.15.12:0/2809129543 shutdown_connections 2023-07-08T12:38:13.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.611+0000 7f6440b65700 1 -- 172.21.15.12:0/2809129543 wait complete. 2023-07-08T12:38:13.615 INFO:teuthology.orchestra.run.smithi012.stderr:added data pool 5 to fsmap 2023-07-08T12:38:13.628 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-08T12:38:13.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.759+0000 7faf1c209700 1 Processor -- start 2023-07-08T12:38:13.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.759+0000 7faf1c209700 1 -- start start 2023-07-08T12:38:13.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.760+0000 7faf1c209700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf14156be0 0x7faf140cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:13.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.760+0000 7faf1c209700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf140cd600 0x7faf140cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:13.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.760+0000 7faf1c209700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf140cda70 con 0x7faf140cb4f0 2023-07-08T12:38:13.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.760+0000 7faf1c209700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf14156fb0 con 0x7faf140cd600 2023-07-08T12:38:13.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.760+0000 7faf1c209700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1415da40 con 0x7faf14156be0 2023-07-08T12:38:13.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf19fa5700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf14156be0 0x7faf140cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:13.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf197a4700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf140cd600 0x7faf140cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:13.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf197a4700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf140cd600 0x7faf140cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:47848/0 (socket says 172.21.15.12:47848) 2023-07-08T12:38:13.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf197a4700 1 -- 172.21.15.12:0/668563275 learned_addr learned my addr 172.21.15.12:0/668563275 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:13.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf1a7a6700 1 --1- 172.21.15.12:0/668563275 >> v1:172.21.15.160:6789/0 conn(0x7faf140cb4f0 0x7faf14070340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:45724/0 (socket says 172.21.15.12:45724) 2023-07-08T12:38:13.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf197a4700 1 -- 172.21.15.12:0/668563275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf14156be0 msgr2=0x7faf140cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:13.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf197a4700 1 --2- 172.21.15.12:0/668563275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf14156be0 0x7faf140cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:13.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf197a4700 1 -- 172.21.15.12:0/668563275 >> v1:172.21.15.160:6789/0 conn(0x7faf140cb4f0 legacy=0x7faf14070340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:13.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf197a4700 1 -- 172.21.15.12:0/668563275 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf140cbc00 con 0x7faf140cd600 2023-07-08T12:38:13.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf18fa3700 1 -- 172.21.15.12:0/668563275 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309419341 0 0) 0x7faf0c001490 con 0x7faf140cb4f0 2023-07-08T12:38:13.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf197a4700 1 --2- 172.21.15.12:0/668563275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf140cd600 0x7faf140cafb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf0400bbc0 tx=0x7faf0400bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8264ccd7462b9b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:13.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf18fa3700 1 -- 172.21.15.12:0/668563275 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0400dcf0 con 0x7faf140cd600 2023-07-08T12:38:13.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf18fa3700 1 -- 172.21.15.12:0/668563275 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0400de50 con 0x7faf140cd600 2023-07-08T12:38:13.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.761+0000 7faf18fa3700 1 -- 172.21.15.12:0/668563275 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04010690 con 0x7faf140cd600 2023-07-08T12:38:13.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.762+0000 7faf1c209700 1 -- 172.21.15.12:0/668563275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf140cd600 msgr2=0x7faf140cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:13.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.762+0000 7faf1c209700 1 --2- 172.21.15.12:0/668563275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf140cd600 0x7faf140cafb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf0400bbc0 tx=0x7faf0400bed0 comp rx=0 tx=0).stop 2023-07-08T12:38:13.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.762+0000 7faf1c209700 1 -- 172.21.15.12:0/668563275 shutdown_connections 2023-07-08T12:38:13.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.762+0000 7faf1c209700 1 --2- 172.21.15.12:0/668563275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf14156be0 0x7faf140cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:13.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.762+0000 7faf1c209700 1 --2- 172.21.15.12:0/668563275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf140cd600 0x7faf140cafb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:13.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.762+0000 7faf1c209700 1 -- 172.21.15.12:0/668563275 >> 172.21.15.12:0/668563275 conn(0x7faf140b0390 msgr2=0x7faf140b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:13.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.762+0000 7faf1c209700 1 -- 172.21.15.12:0/668563275 shutdown_connections 2023-07-08T12:38:13.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.762+0000 7faf1c209700 1 -- 172.21.15.12:0/668563275 wait complete. 2023-07-08T12:38:13.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.764+0000 7faf1c209700 1 Processor -- start 2023-07-08T12:38:13.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.764+0000 7faf1c209700 1 -- start start 2023-07-08T12:38:13.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1c209700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf140cb4f0 0x7faf141536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:13.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1c209700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf140cd600 0x7faf140c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:13.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1c209700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf14156be0 0x7faf140c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:13.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1c209700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf140c6400 con 0x7faf14156be0 2023-07-08T12:38:13.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1c209700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf14153cb0 con 0x7faf140cd600 2023-07-08T12:38:13.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1c209700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf14153f90 con 0x7faf140cb4f0 2023-07-08T12:38:13.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1a7a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf14156be0 0x7faf140c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:13.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1a7a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf14156be0 0x7faf140c4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49112/0 (socket says 172.21.15.12:49112) 2023-07-08T12:38:13.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1a7a6700 1 -- 172.21.15.12:0/29407018 learned_addr learned my addr 172.21.15.12:0/29407018 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:13.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf197a4700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf140cd600 0x7faf140c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:13.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf19fa5700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf140cb4f0 0x7faf141536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:13.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1a7a6700 1 -- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf140cb4f0 msgr2=0x7faf141536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:13.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1a7a6700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf140cb4f0 0x7faf141536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:13.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1a7a6700 1 -- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf140cd600 msgr2=0x7faf140c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:13.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1a7a6700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf140cd600 0x7faf140c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:13.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1a7a6700 1 -- 172.21.15.12:0/29407018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0400b7f0 con 0x7faf14156be0 2023-07-08T12:38:13.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.765+0000 7faf1a7a6700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf14156be0 0x7faf140c4660 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00cca0 tx=0x7faf0c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:13.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.766+0000 7faf0a7fc700 1 -- 172.21.15.12:0/29407018 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00ab10 con 0x7faf14156be0 2023-07-08T12:38:13.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.766+0000 7faf0a7fc700 1 -- 172.21.15.12:0/29407018 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c00ac90 con 0x7faf14156be0 2023-07-08T12:38:13.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.766+0000 7faf0a7fc700 1 -- 172.21.15.12:0/29407018 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0070d0 con 0x7faf14156be0 2023-07-08T12:38:13.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.766+0000 7faf1c209700 1 -- 172.21.15.12:0/29407018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf14154270 con 0x7faf14156be0 2023-07-08T12:38:13.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.766+0000 7faf1c209700 1 -- 172.21.15.12:0/29407018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf140c8640 con 0x7faf14156be0 2023-07-08T12:38:13.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.766+0000 7faf1c209700 1 -- 172.21.15.12:0/29407018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf140c6540 con 0x7faf14156be0 2023-07-08T12:38:13.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.767+0000 7faf0a7fc700 1 -- 172.21.15.12:0/29407018 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faf0c007250 con 0x7faf14156be0 2023-07-08T12:38:13.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.767+0000 7faf0a7fc700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faef806ace0 0x7faef806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:13.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.767+0000 7faf0a7fc700 1 -- 172.21.15.12:0/29407018 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faf0c088fe0 con 0x7faf14156be0 2023-07-08T12:38:13.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.768+0000 7faf19fa5700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faef806ace0 0x7faef806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:13.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.768+0000 7faf19fa5700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faef806ace0 0x7faef806d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faf10009b40 tx=0x7faf10009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:13.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:13.772+0000 7faf0a7fc700 1 -- 172.21.15.12:0/29407018 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faf140c6540 con 0x7faf14156be0 2023-07-08T12:38:14.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.013+0000 7faf1c209700 1 -- 172.21.15.12:0/29407018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf140cda70 con 0x7faf14156be0 2023-07-08T12:38:14.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.014+0000 7faf0a7fc700 1 -- 172.21.15.12:0/29407018 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4670 (secure 0 0 0) 0x7faf140cda70 con 0x7faf14156be0 2023-07-08T12:38:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:14.017 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3238006310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3238006310},{"type":"v1","addr":"172.21.15.160:6836","nonce":3238006310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3864661454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3864661454},{"type":"v1","addr":"172.21.15.160:6837","nonce":3864661454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6834/946118830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":946118830},{"type":"v1","addr":"172.21.15.12:6834","nonce":946118830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-08T12:38:07.943202+0000","modified":"2023-07-08T12:38:13.600773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6835/3301134992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3301134992},{"type":"v1","addr":"172.21.15.12:6835","nonce":3301134992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-08T12:38:14.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.015+0000 7faf1c209700 1 -- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faef806ace0 msgr2=0x7faef806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.015+0000 7faf1c209700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faef806ace0 0x7faef806d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faf10009b40 tx=0x7faf10009210 comp rx=0 tx=0).stop 2023-07-08T12:38:14.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.015+0000 7faf1c209700 1 -- 172.21.15.12:0/29407018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf14156be0 msgr2=0x7faf140c4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.015+0000 7faf1c209700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf14156be0 0x7faf140c4660 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00cca0 tx=0x7faf0c00a060 comp rx=0 tx=0).stop 2023-07-08T12:38:14.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.016+0000 7faf1c209700 1 -- 172.21.15.12:0/29407018 shutdown_connections 2023-07-08T12:38:14.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.016+0000 7faf1c209700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faef806ace0 0x7faef806d190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.016+0000 7faf1c209700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf14156be0 0x7faf140c4660 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.016+0000 7faf1c209700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf140cb4f0 0x7faf141536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.016+0000 7faf1c209700 1 --2- 172.21.15.12:0/29407018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf140cd600 0x7faf140c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.016+0000 7faf1c209700 1 -- 172.21.15.12:0/29407018 >> 172.21.15.12:0/29407018 conn(0x7faf140b0390 msgr2=0x7faf140ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:14.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.016+0000 7faf1c209700 1 -- 172.21.15.12:0/29407018 shutdown_connections 2023-07-08T12:38:14.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.016+0000 7faf1c209700 1 -- 172.21.15.12:0/29407018 wait complete. 2023-07-08T12:38:14.020 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 7 2023-07-08T12:38:14.030 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:14.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.153+0000 7f0fcd18f700 1 Processor -- start 2023-07-08T12:38:14.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.153+0000 7f0fcd18f700 1 -- start start 2023-07-08T12:38:14.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fcd18f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0fc80ce1b0 0x7f0fc80cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fcd18f700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0fc80b1c70 0x7f0fc80ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fcd18f700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc815d230 con 0x7f0fc815d6c0 2023-07-08T12:38:14.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fcd18f700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc80c9e60 con 0x7f0fc80b1c70 2023-07-08T12:38:14.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fcd18f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc80cd920 con 0x7f0fc80ce1b0 2023-07-08T12:38:14.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fc759e700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0fc815d6c0 0x7f0fc8071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36516/0 (socket says 172.21.15.12:36516) 2023-07-08T12:38:14.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fc759e700 1 -- 172.21.15.12:0/3636083099 learned_addr learned my addr 172.21.15.12:0/3636083099 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:14.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fc659c700 1 --2- 172.21.15.12:0/3636083099 >> v2:172.21.15.160:3300/0 conn(0x7f0fc80b1c70 0x7f0fc80ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fc6d9d700 1 --2- 172.21.15.12:0/3636083099 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0fc80ce1b0 0x7f0fc80cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fc5d9b700 1 -- 172.21.15.12:0/3636083099 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880992794 0 0) 0x7f0fc815d230 con 0x7f0fc815d6c0 2023-07-08T12:38:14.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fc5d9b700 1 -- 172.21.15.12:0/3636083099 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fb00057c0 con 0x7f0fc815d6c0 2023-07-08T12:38:14.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fc659c700 1 -- 172.21.15.12:0/3636083099 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0fc80ce1b0 msgr2=0x7f0fc80cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fc659c700 1 --2- 172.21.15.12:0/3636083099 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0fc80ce1b0 0x7f0fc80cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fc659c700 1 -- 172.21.15.12:0/3636083099 >> v1:172.21.15.12:6789/0 conn(0x7f0fc815d6c0 legacy=0x7f0fc8071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fc659c700 1 -- 172.21.15.12:0/3636083099 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc80ceac0 con 0x7f0fc80b1c70 2023-07-08T12:38:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.154+0000 7f0fc659c700 1 --2- 172.21.15.12:0/3636083099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0fc80b1c70 0x7f0fc80ce580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009840 tx=0x7f0fbc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df6f4a05090fabc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:14.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.155+0000 7f0fc5d9b700 1 -- 172.21.15.12:0/3636083099 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc011830 con 0x7f0fc80b1c70 2023-07-08T12:38:14.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.155+0000 7f0fc5d9b700 1 -- 172.21.15.12:0/3636083099 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc011990 con 0x7f0fc80b1c70 2023-07-08T12:38:14.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.155+0000 7f0fc5d9b700 1 -- 172.21.15.12:0/3636083099 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc011d00 con 0x7f0fc80b1c70 2023-07-08T12:38:14.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.155+0000 7f0fcd18f700 1 -- 172.21.15.12:0/3636083099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0fc80b1c70 msgr2=0x7f0fc80ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.155+0000 7f0fcd18f700 1 --2- 172.21.15.12:0/3636083099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0fc80b1c70 0x7f0fc80ce580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009840 tx=0x7f0fbc009b50 comp rx=0 tx=0).stop 2023-07-08T12:38:14.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.155+0000 7f0fcd18f700 1 -- 172.21.15.12:0/3636083099 shutdown_connections 2023-07-08T12:38:14.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.155+0000 7f0fcd18f700 1 --2- 172.21.15.12:0/3636083099 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0fc80ce1b0 0x7f0fc80cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.155+0000 7f0fcd18f700 1 --2- 172.21.15.12:0/3636083099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0fc80b1c70 0x7f0fc80ce580 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.155+0000 7f0fcd18f700 1 -- 172.21.15.12:0/3636083099 >> 172.21.15.12:0/3636083099 conn(0x7f0fc80cc290 msgr2=0x7f0fc80cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:14.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.155+0000 7f0fcd18f700 1 -- 172.21.15.12:0/3636083099 shutdown_connections 2023-07-08T12:38:14.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.155+0000 7f0fcd18f700 1 -- 172.21.15.12:0/3636083099 wait complete. 2023-07-08T12:38:14.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.157+0000 7f0fcd18f700 1 Processor -- start 2023-07-08T12:38:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.157+0000 7f0fcd18f700 1 -- start start 2023-07-08T12:38:14.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.157+0000 7f0fcd18f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0fc80b1c70 0x7f0fc8157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fcd18f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0fc80ce1b0 0x7f0fc81f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fcd18f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc815d6c0 0x7f0fc80c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fcd18f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81f2590 con 0x7f0fc815d6c0 2023-07-08T12:38:14.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fcd18f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81f2870 con 0x7f0fc80b1c70 2023-07-08T12:38:14.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fcd18f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81f2b50 con 0x7f0fc80ce1b0 2023-07-08T12:38:14.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fc759e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc815d6c0 0x7f0fc80c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fc759e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc815d6c0 0x7f0fc80c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49126/0 (socket says 172.21.15.12:49126) 2023-07-08T12:38:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fc759e700 1 -- 172.21.15.12:0/122422232 learned_addr learned my addr 172.21.15.12:0/122422232 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fc6d9d700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0fc80b1c70 0x7f0fc8157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fc659c700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0fc80ce1b0 0x7f0fc81f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fc759e700 1 -- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0fc80ce1b0 msgr2=0x7f0fc81f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fc759e700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0fc80ce1b0 0x7f0fc81f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fc759e700 1 -- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0fc80b1c70 msgr2=0x7f0fc8157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fc759e700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0fc80b1c70 0x7f0fc8157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fc759e700 1 -- 172.21.15.12:0/122422232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc0094a0 con 0x7f0fc815d6c0 2023-07-08T12:38:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fc759e700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc815d6c0 0x7f0fc80c68e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000cb40 tx=0x7f0fc000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0faf7fe700 1 -- 172.21.15.12:0/122422232 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc000a9e0 con 0x7f0fc815d6c0 2023-07-08T12:38:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0faf7fe700 1 -- 172.21.15.12:0/122422232 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc000ab40 con 0x7f0fc815d6c0 2023-07-08T12:38:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0fcd18f700 1 -- 172.21.15.12:0/122422232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc8159860 con 0x7f0fc815d6c0 2023-07-08T12:38:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.158+0000 7f0faf7fe700 1 -- 172.21.15.12:0/122422232 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0012400 con 0x7f0fc815d6c0 2023-07-08T12:38:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.159+0000 7f0fcd18f700 1 -- 172.21.15.12:0/122422232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc8158500 con 0x7f0fc815d6c0 2023-07-08T12:38:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.159+0000 7f0fcd18f700 1 -- 172.21.15.12:0/122422232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc8159d80 con 0x7f0fc815d6c0 2023-07-08T12:38:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.160+0000 7f0faf7fe700 1 -- 172.21.15.12:0/122422232 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0fc001b3b0 con 0x7f0fc815d6c0 2023-07-08T12:38:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.160+0000 7f0faf7fe700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0fb406ace0 0x7f0fb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.160+0000 7f0faf7fe700 1 -- 172.21.15.12:0/122422232 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0fc0089250 con 0x7f0fc815d6c0 2023-07-08T12:38:14.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.161+0000 7f0fc6d9d700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0fb406ace0 0x7f0fb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.161+0000 7f0fc6d9d700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0fb406ace0 0x7f0fb406d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80af910 tx=0x7f0fb8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:14.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.165+0000 7f0faf7fe700 1 -- 172.21.15.12:0/122422232 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0fc8159d80 con 0x7f0fc815d6c0 2023-07-08T12:38:14.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.365+0000 7f0fcd18f700 1 -- 172.21.15.12:0/122422232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fc8159d80 con 0x7f0fc815d6c0 2023-07-08T12:38:14.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.367+0000 7f0faf7fe700 1 -- 172.21.15.12:0/122422232 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f0fc8159d80 con 0x7f0fc815d6c0 2023-07-08T12:38:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:14.370 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:12.993803+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:14.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.369+0000 7f0fcd18f700 1 -- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0fb406ace0 msgr2=0x7f0fb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.369+0000 7f0fcd18f700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0fb406ace0 0x7f0fb406d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80af910 tx=0x7f0fb8009210 comp rx=0 tx=0).stop 2023-07-08T12:38:14.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.369+0000 7f0fcd18f700 1 -- 172.21.15.12:0/122422232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc815d6c0 msgr2=0x7f0fc80c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.369+0000 7f0fcd18f700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc815d6c0 0x7f0fc80c68e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000cb40 tx=0x7f0fc000ce50 comp rx=0 tx=0).stop 2023-07-08T12:38:14.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.369+0000 7f0fcd18f700 1 -- 172.21.15.12:0/122422232 shutdown_connections 2023-07-08T12:38:14.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.369+0000 7f0fcd18f700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0fb406ace0 0x7f0fb406d190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.369+0000 7f0fcd18f700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc815d6c0 0x7f0fc80c68e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.369+0000 7f0fcd18f700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0fc80b1c70 0x7f0fc8157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.369+0000 7f0fcd18f700 1 --2- 172.21.15.12:0/122422232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0fc80ce1b0 0x7f0fc81f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.369+0000 7f0fcd18f700 1 -- 172.21.15.12:0/122422232 >> 172.21.15.12:0/122422232 conn(0x7f0fc80cc290 msgr2=0x7f0fc80cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:14.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.369+0000 7f0fcd18f700 1 -- 172.21.15.12:0/122422232 shutdown_connections 2023-07-08T12:38:14.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.369+0000 7f0fcd18f700 1 -- 172.21.15.12:0/122422232 wait complete. 2023-07-08T12:38:14.384 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:14.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.506+0000 7fd7f3bf7700 1 Processor -- start 2023-07-08T12:38:14.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.507+0000 7fd7f3bf7700 1 -- start start 2023-07-08T12:38:14.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.507+0000 7fd7f3bf7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd7ec0ce1b0 0x7fd7ec0cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.507+0000 7fd7f3bf7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7ec0b1c70 0x7fd7ec0ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.507+0000 7fd7f3bf7700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7ec15d230 con 0x7fd7ec15d6c0 2023-07-08T12:38:14.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.507+0000 7fd7f3bf7700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ec0c9e60 con 0x7fd7ec0ce1b0 2023-07-08T12:38:14.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.507+0000 7fd7f3bf7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec0cd920 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.508+0000 7fd7f1993700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd7ec0ce1b0 0x7fd7ec0cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.508+0000 7fd7f1993700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd7ec0ce1b0 0x7fd7ec0cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49130/0 (socket says 172.21.15.12:49130) 2023-07-08T12:38:14.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.508+0000 7fd7f1993700 1 -- 172.21.15.12:0/385828357 learned_addr learned my addr 172.21.15.12:0/385828357 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:14.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.508+0000 7fd7f1192700 1 --2- 172.21.15.12:0/385828357 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7ec0b1c70 0x7fd7ec0ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.508+0000 7fd7f1993700 1 -- 172.21.15.12:0/385828357 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7ec0b1c70 msgr2=0x7fd7ec0ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:14.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.508+0000 7fd7f1993700 1 --2- 172.21.15.12:0/385828357 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7ec0b1c70 0x7fd7ec0ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.508+0000 7fd7f1993700 1 -- 172.21.15.12:0/385828357 >> v1:172.21.15.160:6789/0 conn(0x7fd7ec15d6c0 legacy=0x7fd7ec071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.508+0000 7fd7f1993700 1 -- 172.21.15.12:0/385828357 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec0ceac0 con 0x7fd7ec0ce1b0 2023-07-08T12:38:14.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.508+0000 7fd7f1993700 1 --2- 172.21.15.12:0/385828357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7ec0ce1b0 0x7fd7ec0cd3e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0098f0 tx=0x7fd7dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832662fe0dc2e97e server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:14.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.508+0000 7fd7f0991700 1 -- 172.21.15.12:0/385828357 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc00b8a0 con 0x7fd7ec0ce1b0 2023-07-08T12:38:14.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.508+0000 7fd7f0991700 1 -- 172.21.15.12:0/385828357 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc00ba00 con 0x7fd7ec0ce1b0 2023-07-08T12:38:14.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.508+0000 7fd7f0991700 1 -- 172.21.15.12:0/385828357 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc00bd90 con 0x7fd7ec0ce1b0 2023-07-08T12:38:14.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.509+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/385828357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7ec0ce1b0 msgr2=0x7fd7ec0cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.509+0000 7fd7f3bf7700 1 --2- 172.21.15.12:0/385828357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7ec0ce1b0 0x7fd7ec0cd3e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0098f0 tx=0x7fd7dc009c00 comp rx=0 tx=0).stop 2023-07-08T12:38:14.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.509+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/385828357 shutdown_connections 2023-07-08T12:38:14.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.509+0000 7fd7f3bf7700 1 --2- 172.21.15.12:0/385828357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7ec0ce1b0 0x7fd7ec0cd3e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.509+0000 7fd7f3bf7700 1 --2- 172.21.15.12:0/385828357 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7ec0b1c70 0x7fd7ec0ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.509+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/385828357 >> 172.21.15.12:0/385828357 conn(0x7fd7ec0cc290 msgr2=0x7fd7ec0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.509+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/385828357 shutdown_connections 2023-07-08T12:38:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.509+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/385828357 wait complete. 2023-07-08T12:38:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.510+0000 7fd7f3bf7700 1 Processor -- start 2023-07-08T12:38:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.510+0000 7fd7f3bf7700 1 -- start start 2023-07-08T12:38:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f3bf7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7ec0b1c70 0x7fd7ec0c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f3bf7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7ec0ce1b0 0x7fd7ec0c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f3bf7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7ec15d6c0 0x7fd7ec15a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f3bf7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec0c4e90 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f3bf7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec154b80 con 0x7fd7ec0ce1b0 2023-07-08T12:38:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f3bf7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec155700 con 0x7fd7ec15d6c0 2023-07-08T12:38:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f1993700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7ec0b1c70 0x7fd7ec0c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f1993700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7ec0b1c70 0x7fd7ec0c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49136/0 (socket says 172.21.15.12:49136) 2023-07-08T12:38:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f1993700 1 -- 172.21.15.12:0/1007885749 learned_addr learned my addr 172.21.15.12:0/1007885749 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f1192700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7ec0ce1b0 0x7fd7ec0c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f2194700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7ec15d6c0 0x7fd7ec15a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f1993700 1 -- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7ec15d6c0 msgr2=0x7fd7ec15a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f1993700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7ec15d6c0 0x7fd7ec15a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f1993700 1 -- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7ec0ce1b0 msgr2=0x7fd7ec0c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f1993700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7ec0ce1b0 0x7fd7ec0c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f1993700 1 -- 172.21.15.12:0/1007885749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc009550 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f1993700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7ec0b1c70 0x7fd7ec0c4060 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc000c00 tx=0x7fd7dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:14.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7e27fc700 1 -- 172.21.15.12:0/1007885749 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc004510 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7e27fc700 1 -- 172.21.15.12:0/1007885749 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc01e070 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/1007885749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ec154dd0 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.511+0000 7fd7e27fc700 1 -- 172.21.15.12:0/1007885749 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc0038e0 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.512+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/1007885749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ec155390 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.512+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/1007885749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ec0cef20 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.513+0000 7fd7e27fc700 1 -- 172.21.15.12:0/1007885749 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd7dc003a40 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.513+0000 7fd7e27fc700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd7d006ace0 0x7fd7d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.513+0000 7fd7e27fc700 1 -- 172.21.15.12:0/1007885749 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd7dc020070 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.513+0000 7fd7f1192700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd7d006ace0 0x7fd7d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.514+0000 7fd7f1192700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd7d006ace0 0x7fd7d006d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec1530d0 tx=0x7fd7e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:14.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.518+0000 7fd7e27fc700 1 -- 172.21.15.12:0/1007885749 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd7dc0558b0 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.720+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/1007885749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7ec15ce90 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.721+0000 7fd7e27fc700 1 -- 172.21.15.12:0/1007885749 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20108 (secure 0 0 0) 0x7fd7dc058ed0 con 0x7fd7ec0b1c70 2023-07-08T12:38:14.724 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:14.724 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:12.993803+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.723+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd7d006ace0 msgr2=0x7fd7d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.723+0000 7fd7f3bf7700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd7d006ace0 0x7fd7d006d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec1530d0 tx=0x7fd7e8008040 comp rx=0 tx=0).stop 2023-07-08T12:38:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.723+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/1007885749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7ec0b1c70 msgr2=0x7fd7ec0c4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.723+0000 7fd7f3bf7700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7ec0b1c70 0x7fd7ec0c4060 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc000c00 tx=0x7fd7dc004750 comp rx=0 tx=0).stop 2023-07-08T12:38:14.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.724+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/1007885749 shutdown_connections 2023-07-08T12:38:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.724+0000 7fd7f3bf7700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd7d006ace0 0x7fd7d006d190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.724+0000 7fd7f3bf7700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7ec15d6c0 0x7fd7ec15a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.724+0000 7fd7f3bf7700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7ec0b1c70 0x7fd7ec0c4060 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.724+0000 7fd7f3bf7700 1 --2- 172.21.15.12:0/1007885749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7ec0ce1b0 0x7fd7ec0c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.724+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/1007885749 >> 172.21.15.12:0/1007885749 conn(0x7fd7ec0cc290 msgr2=0x7fd7ec0cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.724+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/1007885749 shutdown_connections 2023-07-08T12:38:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.724+0000 7fd7f3bf7700 1 -- 172.21.15.12:0/1007885749 wait complete. 2023-07-08T12:38:14.765 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-08T12:38:14.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.887+0000 7fd6188ff700 1 Processor -- start 2023-07-08T12:38:14.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.888+0000 7fd6188ff700 1 -- start start 2023-07-08T12:38:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.888+0000 7fd6188ff700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd614160790 0x7fd6140cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.888+0000 7fd6188ff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd614069f30 0x7fd61406a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.888+0000 7fd6188ff700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61415c890 con 0x7fd614159c30 2023-07-08T12:38:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.888+0000 7fd6188ff700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61415c730 con 0x7fd614069f30 2023-07-08T12:38:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.888+0000 7fd6188ff700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61406a910 con 0x7fd614160790 2023-07-08T12:38:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.889+0000 7fd612d9d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd614069f30 0x7fd61406a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.889+0000 7fd61259c700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd614159c30 0x7fd61406ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36540/0 (socket says 172.21.15.12:36540) 2023-07-08T12:38:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.889+0000 7fd61259c700 1 -- 172.21.15.12:0/752448890 learned_addr learned my addr 172.21.15.12:0/752448890 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.889+0000 7fd611d9b700 1 --2- 172.21.15.12:0/752448890 >> v2:172.21.15.160:3300/0 conn(0x7fd614160790 0x7fd6140cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.889+0000 7fd612d9d700 1 -- 172.21.15.12:0/752448890 >> v2:172.21.15.160:3300/0 conn(0x7fd614160790 msgr2=0x7fd6140cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.889+0000 7fd612d9d700 1 --2- 172.21.15.12:0/752448890 >> v2:172.21.15.160:3300/0 conn(0x7fd614160790 0x7fd6140cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.889+0000 7fd612d9d700 1 -- 172.21.15.12:0/752448890 >> v1:172.21.15.12:6789/0 conn(0x7fd614159c30 legacy=0x7fd61406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.889+0000 7fd612d9d700 1 -- 172.21.15.12:0/752448890 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61406f620 con 0x7fd614069f30 2023-07-08T12:38:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.889+0000 7fd612d9d700 1 --2- 172.21.15.12:0/752448890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd614069f30 0x7fd61406a3d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd6140b2370 tx=0x7fd60c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5861c16f123f1857 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.889+0000 7fd61159a700 1 -- 172.21.15.12:0/752448890 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00e070 con 0x7fd614069f30 2023-07-08T12:38:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.889+0000 7fd61159a700 1 -- 172.21.15.12:0/752448890 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c00bbd0 con 0x7fd614069f30 2023-07-08T12:38:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.889+0000 7fd61159a700 1 -- 172.21.15.12:0/752448890 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c012430 con 0x7fd614069f30 2023-07-08T12:38:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.890+0000 7fd6188ff700 1 -- 172.21.15.12:0/752448890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd614069f30 msgr2=0x7fd61406a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.890+0000 7fd6188ff700 1 --2- 172.21.15.12:0/752448890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd614069f30 0x7fd61406a3d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd6140b2370 tx=0x7fd60c00b330 comp rx=0 tx=0).stop 2023-07-08T12:38:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.890+0000 7fd6188ff700 1 -- 172.21.15.12:0/752448890 shutdown_connections 2023-07-08T12:38:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.890+0000 7fd6188ff700 1 --2- 172.21.15.12:0/752448890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd614069f30 0x7fd61406a3d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.890+0000 7fd6188ff700 1 --2- 172.21.15.12:0/752448890 >> v2:172.21.15.160:3300/0 conn(0x7fd614160790 0x7fd6140cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.890+0000 7fd6188ff700 1 -- 172.21.15.12:0/752448890 >> 172.21.15.12:0/752448890 conn(0x7fd6140cc290 msgr2=0x7fd6140cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.890+0000 7fd6188ff700 1 -- 172.21.15.12:0/752448890 shutdown_connections 2023-07-08T12:38:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.890+0000 7fd6188ff700 1 -- 172.21.15.12:0/752448890 wait complete. 2023-07-08T12:38:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.892+0000 7fd6188ff700 1 Processor -- start 2023-07-08T12:38:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.892+0000 7fd6188ff700 1 -- start start 2023-07-08T12:38:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd6188ff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd614069f30 0x7fd614151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd6188ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd614159c30 0x7fd614150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd6188ff700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd614160790 0x7fd6141577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd6188ff700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6141e6a50 con 0x7fd614159c30 2023-07-08T12:38:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd6188ff700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6141e5b60 con 0x7fd614069f30 2023-07-08T12:38:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd6188ff700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6141e5e40 con 0x7fd614160790 2023-07-08T12:38:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd611d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd614159c30 0x7fd614150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd611d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd614159c30 0x7fd614150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49152/0 (socket says 172.21.15.12:49152) 2023-07-08T12:38:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd611d9b700 1 -- 172.21.15.12:0/827362324 learned_addr learned my addr 172.21.15.12:0/827362324 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd61259c700 1 --2- 172.21.15.12:0/827362324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd614069f30 0x7fd614151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd611d9b700 1 -- 172.21.15.12:0/827362324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd614160790 msgr2=0x7fd6141577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd611d9b700 1 --2- 172.21.15.12:0/827362324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd614160790 0x7fd6141577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd611d9b700 1 -- 172.21.15.12:0/827362324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd614069f30 msgr2=0x7fd614151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd611d9b700 1 --2- 172.21.15.12:0/827362324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd614069f30 0x7fd614151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd611d9b700 1 -- 172.21.15.12:0/827362324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c009a70 con 0x7fd614159c30 2023-07-08T12:38:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd611d9b700 1 --2- 172.21.15.12:0/827362324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd614159c30 0x7fd614150390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00e810 tx=0x7fd5fc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd60affd700 1 -- 172.21.15.12:0/827362324 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc013610 con 0x7fd614159c30 2023-07-08T12:38:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd60affd700 1 -- 172.21.15.12:0/827362324 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5fc013770 con 0x7fd614159c30 2023-07-08T12:38:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd6188ff700 1 -- 172.21.15.12:0/827362324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6141e6120 con 0x7fd614159c30 2023-07-08T12:38:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd60affd700 1 -- 172.21.15.12:0/827362324 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc013b00 con 0x7fd614159c30 2023-07-08T12:38:14.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.893+0000 7fd6188ff700 1 -- 172.21.15.12:0/827362324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6141e66e0 con 0x7fd614159c30 2023-07-08T12:38:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.894+0000 7fd6188ff700 1 -- 172.21.15.12:0/827362324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61404ea60 con 0x7fd614159c30 2023-07-08T12:38:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.895+0000 7fd60affd700 1 -- 172.21.15.12:0/827362324 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd5fc013c60 con 0x7fd614159c30 2023-07-08T12:38:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.895+0000 7fd60affd700 1 --2- 172.21.15.12:0/827362324 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd5f406ace0 0x7fd5f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.895+0000 7fd60affd700 1 -- 172.21.15.12:0/827362324 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd5fc089400 con 0x7fd614159c30 2023-07-08T12:38:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.895+0000 7fd61259c700 1 --2- 172.21.15.12:0/827362324 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd5f406ace0 0x7fd5f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.896+0000 7fd61259c700 1 --2- 172.21.15.12:0/827362324 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd5f406ace0 0x7fd5f406d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd604005910 tx=0x7fd604009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:14.899+0000 7fd60affd700 1 -- 172.21.15.12:0/827362324 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd5fc057c50 con 0x7fd614159c30 2023-07-08T12:38:15.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.145+0000 7fd6188ff700 1 -- 172.21.15.12:0/827362324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6140b5790 con 0x7fd614159c30 2023-07-08T12:38:15.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.146+0000 7fd60affd700 1 -- 172.21.15.12:0/827362324 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fd5fc0577e0 con 0x7fd614159c30 2023-07-08T12:38:15.149 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:15.149 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3238006310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3238006310},{"type":"v1","addr":"172.21.15.160:6836","nonce":3238006310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3864661454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3864661454},{"type":"v1","addr":"172.21.15.160:6837","nonce":3864661454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6834/946118830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":946118830},{"type":"v1","addr":"172.21.15.12:6834","nonce":946118830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-08T12:38:07.943202+0000","modified":"2023-07-08T12:38:13.600773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6835/3301134992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3301134992},{"type":"v1","addr":"172.21.15.12:6835","nonce":3301134992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-08T12:38:15.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.148+0000 7fd6188ff700 1 -- 172.21.15.12:0/827362324 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd5f406ace0 msgr2=0x7fd5f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:15.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.148+0000 7fd6188ff700 1 --2- 172.21.15.12:0/827362324 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd5f406ace0 0x7fd5f406d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd604005910 tx=0x7fd604009040 comp rx=0 tx=0).stop 2023-07-08T12:38:15.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.148+0000 7fd6188ff700 1 -- 172.21.15.12:0/827362324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd614159c30 msgr2=0x7fd614150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:15.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.148+0000 7fd6188ff700 1 --2- 172.21.15.12:0/827362324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd614159c30 0x7fd614150390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00e810 tx=0x7fd5fc00ebd0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.148+0000 7fd6188ff700 1 -- 172.21.15.12:0/827362324 shutdown_connections 2023-07-08T12:38:15.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.148+0000 7fd6188ff700 1 --2- 172.21.15.12:0/827362324 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd5f406ace0 0x7fd5f406d190 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.148+0000 7fd6188ff700 1 --2- 172.21.15.12:0/827362324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd614160790 0x7fd6141577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.148+0000 7fd6188ff700 1 --2- 172.21.15.12:0/827362324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd614069f30 0x7fd614151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.148+0000 7fd6188ff700 1 --2- 172.21.15.12:0/827362324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd614159c30 0x7fd614150390 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.148+0000 7fd6188ff700 1 -- 172.21.15.12:0/827362324 >> 172.21.15.12:0/827362324 conn(0x7fd6140cc290 msgr2=0x7fd61406f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:15.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.148+0000 7fd6188ff700 1 -- 172.21.15.12:0/827362324 shutdown_connections 2023-07-08T12:38:15.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.148+0000 7fd6188ff700 1 -- 172.21.15.12:0/827362324 wait complete. 2023-07-08T12:38:15.155 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 7 2023-07-08T12:38:15.163 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:15.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.286+0000 7f280b453700 1 Processor -- start 2023-07-08T12:38:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.286+0000 7f280b453700 1 -- start start 2023-07-08T12:38:15.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.287+0000 7f280b453700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28040ce1b0 0x7f28040cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:15.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.287+0000 7f280b453700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f28040b1c70 0x7f28040ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:15.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.287+0000 7f280b453700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f280415d6c0 0x7f2804071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:15.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.287+0000 7f280b453700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28040c9e60 con 0x7f280415d6c0 2023-07-08T12:38:15.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.287+0000 7f280b453700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280415d230 con 0x7f28040b1c70 2023-07-08T12:38:15.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.287+0000 7f280b453700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040cd920 con 0x7f28040ce1b0 2023-07-08T12:38:15.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.287+0000 7f28099f0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f280415d6c0 0x7f2804071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:15.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.287+0000 7f28099f0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f280415d6c0 0x7f2804071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49162/0 (socket says 172.21.15.12:49162) 2023-07-08T12:38:15.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.287+0000 7f28099f0700 1 -- 172.21.15.12:0/249363197 learned_addr learned my addr 172.21.15.12:0/249363197 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:15.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.287+0000 7f28089ee700 1 --2- 172.21.15.12:0/249363197 >> v2:172.21.15.160:3300/0 conn(0x7f28040b1c70 0x7f28040ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:15.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.287+0000 7f28091ef700 1 --2- 172.21.15.12:0/249363197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28040ce1b0 0x7f28040cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:15.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.288+0000 7f28099f0700 1 -- 172.21.15.12:0/249363197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28040ce1b0 msgr2=0x7f28040cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:15.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.288+0000 7f28099f0700 1 --2- 172.21.15.12:0/249363197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28040ce1b0 0x7f28040cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.288+0000 7f28099f0700 1 -- 172.21.15.12:0/249363197 >> v2:172.21.15.160:3300/0 conn(0x7f28040b1c70 msgr2=0x7f28040ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.288+0000 7f28099f0700 1 --2- 172.21.15.12:0/249363197 >> v2:172.21.15.160:3300/0 conn(0x7f28040b1c70 0x7f28040ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.288+0000 7f28099f0700 1 -- 172.21.15.12:0/249363197 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28040ceac0 con 0x7f280415d6c0 2023-07-08T12:38:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.288+0000 7f28099f0700 1 --2- 172.21.15.12:0/249363197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f280415d6c0 0x7f2804071530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2804071220 tx=0x7f27fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f688494980e99ac3 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.288+0000 7f27fbfff700 1 -- 172.21.15.12:0/249363197 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc00e070 con 0x7f280415d6c0 2023-07-08T12:38:15.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.288+0000 7f27fbfff700 1 -- 172.21.15.12:0/249363197 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc00b910 con 0x7f280415d6c0 2023-07-08T12:38:15.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.288+0000 7f27fbfff700 1 -- 172.21.15.12:0/249363197 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc00bcc0 con 0x7f280415d6c0 2023-07-08T12:38:15.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.289+0000 7f280b453700 1 -- 172.21.15.12:0/249363197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f280415d6c0 msgr2=0x7f2804071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:15.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.289+0000 7f280b453700 1 --2- 172.21.15.12:0/249363197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f280415d6c0 0x7f2804071530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2804071220 tx=0x7f27fc00b330 comp rx=0 tx=0).stop 2023-07-08T12:38:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.289+0000 7f280b453700 1 -- 172.21.15.12:0/249363197 shutdown_connections 2023-07-08T12:38:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.289+0000 7f280b453700 1 --2- 172.21.15.12:0/249363197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28040ce1b0 0x7f28040cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.289+0000 7f280b453700 1 --2- 172.21.15.12:0/249363197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f280415d6c0 0x7f2804071530 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.289+0000 7f280b453700 1 --2- 172.21.15.12:0/249363197 >> v2:172.21.15.160:3300/0 conn(0x7f28040b1c70 0x7f28040ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.289+0000 7f280b453700 1 -- 172.21.15.12:0/249363197 >> 172.21.15.12:0/249363197 conn(0x7f28040cc290 msgr2=0x7f28040cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.289+0000 7f280b453700 1 -- 172.21.15.12:0/249363197 shutdown_connections 2023-07-08T12:38:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.289+0000 7f280b453700 1 -- 172.21.15.12:0/249363197 wait complete. 2023-07-08T12:38:15.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.290+0000 7f280b453700 1 Processor -- start 2023-07-08T12:38:15.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.290+0000 7f280b453700 1 -- start start 2023-07-08T12:38:15.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f280b453700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28040b1c70 0x7f28040c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:15.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f280b453700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28040ce1b0 0x7f28040c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:15.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f280b453700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f280415d6c0 0x7f28040c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:15.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f280b453700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040c7d60 con 0x7f280415d6c0 2023-07-08T12:38:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f280b453700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040c70e0 con 0x7f28040ce1b0 2023-07-08T12:38:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f280b453700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040c73c0 con 0x7f28040b1c70 2023-07-08T12:38:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f28099f0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f280415d6c0 0x7f28040c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f28099f0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f280415d6c0 0x7f28040c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49168/0 (socket says 172.21.15.12:49168) 2023-07-08T12:38:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f28099f0700 1 -- 172.21.15.12:0/537481123 learned_addr learned my addr 172.21.15.12:0/537481123 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f28091ef700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28040b1c70 0x7f28040c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f28089ee700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28040ce1b0 0x7f28040c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:15.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f28099f0700 1 -- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28040b1c70 msgr2=0x7f28040c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:15.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f28099f0700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28040b1c70 0x7f28040c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f28099f0700 1 -- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28040ce1b0 msgr2=0x7f28040c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:15.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f28099f0700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28040ce1b0 0x7f28040c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f28099f0700 1 -- 172.21.15.12:0/537481123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc009b30 con 0x7f280415d6c0 2023-07-08T12:38:15.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f28089ee700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28040ce1b0 0x7f28040c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:15.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f28099f0700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f280415d6c0 0x7f28040c8180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2804070c60 tx=0x7f27fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:15.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f27f9ffb700 1 -- 172.21.15.12:0/537481123 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc00ce80 con 0x7f280415d6c0 2023-07-08T12:38:15.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f27f9ffb700 1 -- 172.21.15.12:0/537481123 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc004250 con 0x7f280415d6c0 2023-07-08T12:38:15.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f280b453700 1 -- 172.21.15.12:0/537481123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28040c76a0 con 0x7f280415d6c0 2023-07-08T12:38:15.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.291+0000 7f27f9ffb700 1 -- 172.21.15.12:0/537481123 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc004600 con 0x7f280415d6c0 2023-07-08T12:38:15.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.292+0000 7f280b453700 1 -- 172.21.15.12:0/537481123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28040c6630 con 0x7f280415d6c0 2023-07-08T12:38:15.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.292+0000 7f280b453700 1 -- 172.21.15.12:0/537481123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28040c7ea0 con 0x7f280415d6c0 2023-07-08T12:38:15.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.293+0000 7f27f9ffb700 1 -- 172.21.15.12:0/537481123 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f27fc004760 con 0x7f280415d6c0 2023-07-08T12:38:15.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.293+0000 7f27f9ffb700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f27e806ace0 0x7f27e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:15.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.294+0000 7f27f9ffb700 1 -- 172.21.15.12:0/537481123 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f27fc08ac00 con 0x7f280415d6c0 2023-07-08T12:38:15.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.294+0000 7f28091ef700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f27e806ace0 0x7f27e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:15.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.294+0000 7f28091ef700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f27e806ace0 0x7f27e806d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009a10 tx=0x7f27f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:15.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.298+0000 7f27f9ffb700 1 -- 172.21.15.12:0/537481123 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f28040c7ea0 con 0x7f280415d6c0 2023-07-08T12:38:15.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.498+0000 7f280b453700 1 -- 172.21.15.12:0/537481123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28040c7ea0 con 0x7f280415d6c0 2023-07-08T12:38:15.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.500+0000 7f27f9ffb700 1 -- 172.21.15.12:0/537481123 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f28040c7ea0 con 0x7f280415d6c0 2023-07-08T12:38:15.502 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:15.502 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:12.993803+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:15.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.502+0000 7f280b453700 1 -- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f27e806ace0 msgr2=0x7f27e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:15.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.502+0000 7f280b453700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f27e806ace0 0x7f27e806d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009a10 tx=0x7f27f4008040 comp rx=0 tx=0).stop 2023-07-08T12:38:15.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.502+0000 7f280b453700 1 -- 172.21.15.12:0/537481123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f280415d6c0 msgr2=0x7f28040c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:15.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.502+0000 7f280b453700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f280415d6c0 0x7f28040c8180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2804070c60 tx=0x7f27fc004ad0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.502+0000 7f280b453700 1 -- 172.21.15.12:0/537481123 shutdown_connections 2023-07-08T12:38:15.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.502+0000 7f280b453700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f27e806ace0 0x7f27e806d190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.502+0000 7f280b453700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f280415d6c0 0x7f28040c8180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.502+0000 7f280b453700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28040b1c70 0x7f28040c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.502+0000 7f280b453700 1 --2- 172.21.15.12:0/537481123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28040ce1b0 0x7f28040c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:15.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.502+0000 7f280b453700 1 -- 172.21.15.12:0/537481123 >> 172.21.15.12:0/537481123 conn(0x7f28040cc290 msgr2=0x7f28040b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:15.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.502+0000 7f280b453700 1 -- 172.21.15.12:0/537481123 shutdown_connections 2023-07-08T12:38:15.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:15.502+0000 7f280b453700 1 -- 172.21.15.12:0/537481123 wait complete. 2023-07-08T12:38:15.919 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-08T12:38:07.943202+0000', 'modified': '2023-07-08T12:38:13.600773+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4290}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4290': {'gid': 4290, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.12:6835/3301134992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6833', 'nonce': 3301134992}, {'type': 'v1', 'addr': '172.21.15.12:6835', 'nonce': 3301134992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-07-08T12:38:15.919 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-08T12:38:15.919 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-08T12:38:16.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.051+0000 7eff448de700 1 Processor -- start 2023-07-08T12:38:16.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.051+0000 7eff448de700 1 -- start start 2023-07-08T12:38:16.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.052+0000 7eff448de700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7eff4015a3a0 0x7eff4015e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.052+0000 7eff448de700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4015a770 con 0x7eff400c9e60 2023-07-08T12:38:16.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.052+0000 7eff448de700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4015e500 con 0x7eff400ca2e0 2023-07-08T12:38:16.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.052+0000 7eff448de700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff400cbd10 con 0x7eff4015a3a0 2023-07-08T12:38:16.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.052+0000 7eff3dd9b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7eff400c9e60 0x7eff40069f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36568/0 (socket says 172.21.15.12:36568) 2023-07-08T12:38:16.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.052+0000 7eff3dd9b700 1 -- 172.21.15.12:0/2391792384 learned_addr learned my addr 172.21.15.12:0/2391792384 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:16.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.052+0000 7eff3e59c700 1 --2- 172.21.15.12:0/2391792384 >> v2:172.21.15.160:3300/0 conn(0x7eff4015a3a0 0x7eff4015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.052+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806807764 0 0) 0x7eff30001110 con 0x7eff400c9e60 2023-07-08T12:38:16.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.052+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff280035b0 con 0x7eff400c9e60 2023-07-08T12:38:16.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225285006 0 0) 0x7eff4015e500 con 0x7eff400ca2e0 2023-07-08T12:38:16.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff30001110 con 0x7eff400ca2e0 2023-07-08T12:38:16.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2857459826 0 0) 0x7eff280035b0 con 0x7eff400c9e60 2023-07-08T12:38:16.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 >> v2:172.21.15.160:3300/0 conn(0x7eff4015a3a0 msgr2=0x7eff4015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:16.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 --2- 172.21.15.12:0/2391792384 >> v2:172.21.15.160:3300/0 conn(0x7eff4015a3a0 0x7eff4015e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 >> v1:172.21.15.160:6789/0 conn(0x7eff400ca2e0 legacy=0x7eff4006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4015deb0 con 0x7eff400c9e60 2023-07-08T12:38:16.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7eff30001800 con 0x7eff400c9e60 2023-07-08T12:38:16.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 >> v1:172.21.15.12:6789/0 conn(0x7eff400c9e60 legacy=0x7eff40069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 --2- 172.21.15.12:0/2391792384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff28003be0 0x7eff28003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 --2- 172.21.15.12:0/2391792384 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff28004850 0x7eff28004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 --2- 172.21.15.12:0/2391792384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff28005200 0x7eff2800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff400cbd10 con 0x7eff28003be0 2023-07-08T12:38:16.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2800c0e0 con 0x7eff28005200 2023-07-08T12:38:16.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2800c270 con 0x7eff28004850 2023-07-08T12:38:16.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3e59c700 1 --2- 172.21.15.12:0/2391792384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff28003be0 0x7eff28003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/2391792384 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff28004850 0x7eff28004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3ed9d700 1 --2- 172.21.15.12:0/2391792384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff28005200 0x7eff2800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3e59c700 1 -- 172.21.15.12:0/2391792384 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff28004850 msgr2=0x7eff28004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:16.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3e59c700 1 --2- 172.21.15.12:0/2391792384 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff28004850 0x7eff28004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3e59c700 1 -- 172.21.15.12:0/2391792384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff28005200 msgr2=0x7eff2800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:16.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3e59c700 1 --2- 172.21.15.12:0/2391792384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff28005200 0x7eff2800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3e59c700 1 -- 172.21.15.12:0/2391792384 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff280035b0 con 0x7eff28003be0 2023-07-08T12:38:16.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3ed9d700 1 --2- 172.21.15.12:0/2391792384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff28005200 0x7eff2800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:16.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3ed9d700 1 -- 172.21.15.12:0/2391792384 reap_dead start 2023-07-08T12:38:16.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3e59c700 1 --2- 172.21.15.12:0/2391792384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff28003be0 0x7eff28003fd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff34009b30 tx=0x7eff28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d711d490caf11203 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:16.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3400e6c0 con 0x7eff28003be0 2023-07-08T12:38:16.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3400e820 con 0x7eff28003be0 2023-07-08T12:38:16.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.053+0000 7eff3d59a700 1 -- 172.21.15.12:0/2391792384 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3400eb90 con 0x7eff28003be0 2023-07-08T12:38:16.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.054+0000 7eff448de700 1 -- 172.21.15.12:0/2391792384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff28003be0 msgr2=0x7eff28003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.054+0000 7eff448de700 1 --2- 172.21.15.12:0/2391792384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff28003be0 0x7eff28003fd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff34009b30 tx=0x7eff28005880 comp rx=0 tx=0).stop 2023-07-08T12:38:16.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.054+0000 7eff448de700 1 -- 172.21.15.12:0/2391792384 shutdown_connections 2023-07-08T12:38:16.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.054+0000 7eff448de700 1 --2- 172.21.15.12:0/2391792384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff28003be0 0x7eff28003fd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.054+0000 7eff448de700 1 -- 172.21.15.12:0/2391792384 >> 172.21.15.12:0/2391792384 conn(0x7eff400cc290 msgr2=0x7eff400cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:16.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.054+0000 7eff448de700 1 -- 172.21.15.12:0/2391792384 shutdown_connections 2023-07-08T12:38:16.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.054+0000 7eff448de700 1 -- 172.21.15.12:0/2391792384 wait complete. 2023-07-08T12:38:16.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.056+0000 7eff448de700 1 Processor -- start 2023-07-08T12:38:16.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.056+0000 7eff448de700 1 -- start start 2023-07-08T12:38:16.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff448de700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff28003be0 0x7eff400c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff448de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff40151be0 0x7eff401513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff448de700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff401e5030 0x7eff400b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff448de700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff401e4020 con 0x7eff40151be0 2023-07-08T12:38:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff448de700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff401e4ba0 con 0x7eff401e5030 2023-07-08T12:38:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff448de700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff40152460 con 0x7eff28003be0 2023-07-08T12:38:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff3dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff40151be0 0x7eff401513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff3dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff40151be0 0x7eff401513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49184/0 (socket says 172.21.15.12:49184) 2023-07-08T12:38:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff3dd9b700 1 -- 172.21.15.12:0/4070850868 learned_addr learned my addr 172.21.15.12:0/4070850868 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff3ed9d700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff401e5030 0x7eff400b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff3e59c700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff28003be0 0x7eff400c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff3dd9b700 1 -- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff28003be0 msgr2=0x7eff400c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff28003be0 0x7eff400c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff3dd9b700 1 -- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff401e5030 msgr2=0x7eff400b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff401e5030 0x7eff400b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff3dd9b700 1 -- 172.21.15.12:0/4070850868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff401e4270 con 0x7eff40151be0 2023-07-08T12:38:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff3e59c700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff28003be0 0x7eff400c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff40151be0 0x7eff401513a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7eff3000e870 tx=0x7eff3000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.057+0000 7eff2effd700 1 -- 172.21.15.12:0/4070850868 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3000c5f0 con 0x7eff40151be0 2023-07-08T12:38:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.058+0000 7eff2effd700 1 -- 172.21.15.12:0/4070850868 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3000c750 con 0x7eff40151be0 2023-07-08T12:38:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.058+0000 7eff448de700 1 -- 172.21.15.12:0/4070850868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff401e4550 con 0x7eff40151be0 2023-07-08T12:38:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.058+0000 7eff2effd700 1 -- 172.21.15.12:0/4070850868 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3000cb00 con 0x7eff40151be0 2023-07-08T12:38:16.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.058+0000 7eff448de700 1 -- 172.21.15.12:0/4070850868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff401518e0 con 0x7eff40151be0 2023-07-08T12:38:16.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.059+0000 7eff2effd700 1 -- 172.21.15.12:0/4070850868 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff3000cc60 con 0x7eff40151be0 2023-07-08T12:38:16.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.059+0000 7eff2effd700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7eff2006ace0 0x7eff2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.060+0000 7eff2effd700 1 -- 172.21.15.12:0/4070850868 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7eff30089380 con 0x7eff40151be0 2023-07-08T12:38:16.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.060+0000 7eff3e59c700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7eff2006ace0 0x7eff2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.060+0000 7eff3e59c700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7eff2006ace0 0x7eff2006d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff340090b0 tx=0x7eff34008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.061+0000 7eff50a63b80 1 Processor -- start 2023-07-08T12:38:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.062+0000 7eff50a63b80 1 -- start start 2023-07-08T12:38:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.062+0000 7eff50a63b80 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac570 0x5638d55cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.062+0000 7eff50a63b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d5589650 0x5638d55ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.062+0000 7eff50a63b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d55ac940 0x5638d55b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.062+0000 7eff50a63b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d548dc60 con 0x5638d5589650 2023-07-08T12:38:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.062+0000 7eff50a63b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d54e09f0 con 0x5638d55ac940 2023-07-08T12:38:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.062+0000 7eff50a63b80 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638d54d9e80 con 0x5638d55ac570 2023-07-08T12:38:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.062+0000 7eff3dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d5589650 0x5638d55ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.062+0000 7eff3dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d5589650 0x5638d55ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49192/0 (socket says 172.21.15.12:49192) 2023-07-08T12:38:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.062+0000 7eff3dd9b700 1 -- 172.21.15.12:0/987112373 learned_addr learned my addr 172.21.15.12:0/987112373 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.063+0000 7eff3e59c700 1 --2- 172.21.15.12:0/987112373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac570 0x5638d55cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.063+0000 7eff3ed9d700 1 --2- 172.21.15.12:0/987112373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d55ac940 0x5638d55b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.063+0000 7eff3dd9b700 1 -- 172.21.15.12:0/987112373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac570 msgr2=0x5638d55cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.063+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/987112373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac570 0x5638d55cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.063+0000 7eff3dd9b700 1 -- 172.21.15.12:0/987112373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d55ac940 msgr2=0x5638d55b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.063+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/987112373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d55ac940 0x5638d55b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.063+0000 7eff3dd9b700 1 -- 172.21.15.12:0/987112373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638d5307d90 con 0x5638d5589650 2023-07-08T12:38:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.063+0000 7eff3ed9d700 1 --2- 172.21.15.12:0/987112373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d55ac940 0x5638d55b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.063+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/987112373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d5589650 0x5638d55ce160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff30087460 tx=0x7eff30057b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af36e7903e60608a server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.063+0000 7eff1ffff700 1 -- 172.21.15.12:0/987112373 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30056230 con 0x5638d5589650 2023-07-08T12:38:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.063+0000 7eff1ffff700 1 -- 172.21.15.12:0/987112373 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff30057380 con 0x5638d5589650 2023-07-08T12:38:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.063+0000 7eff1ffff700 1 -- 172.21.15.12:0/987112373 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3005fd50 con 0x5638d5589650 2023-07-08T12:38:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.064+0000 7eff50a63b80 1 -- 172.21.15.12:0/987112373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d5589650 msgr2=0x5638d55ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.064+0000 7eff50a63b80 1 --2- 172.21.15.12:0/987112373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d5589650 0x5638d55ce160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff30087460 tx=0x7eff30057b00 comp rx=0 tx=0).stop 2023-07-08T12:38:16.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.064+0000 7eff50a63b80 1 -- 172.21.15.12:0/987112373 shutdown_connections 2023-07-08T12:38:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.064+0000 7eff50a63b80 1 --2- 172.21.15.12:0/987112373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d55ac940 0x5638d55b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.064+0000 7eff50a63b80 1 --2- 172.21.15.12:0/987112373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac570 0x5638d55cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.064+0000 7eff50a63b80 1 --2- 172.21.15.12:0/987112373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d5589650 0x5638d55ce160 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.064+0000 7eff50a63b80 1 -- 172.21.15.12:0/987112373 >> 172.21.15.12:0/987112373 conn(0x5638d557da90 msgr2=0x5638d5583010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.064+0000 7eff50a63b80 1 -- 172.21.15.12:0/987112373 shutdown_connections 2023-07-08T12:38:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.064+0000 7eff50a63b80 1 -- 172.21.15.12:0/987112373 wait complete. 2023-07-08T12:38:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.064+0000 7eff50a63b80 1 -- start start 2023-07-08T12:38:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.064+0000 7eff50a63b80 1 Processor -- start 2023-07-08T12:38:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.065+0000 7eff50a63b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d5589650 0x5638d55aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.065+0000 7eff50a63b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 0x5638d55c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.065+0000 7eff50a63b80 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac940 0x5638d55c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.065+0000 7eff50a63b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d5479a70 con 0x5638d55ac570 2023-07-08T12:38:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.065+0000 7eff50a63b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d5479770 con 0x5638d5589650 2023-07-08T12:38:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.065+0000 7eff50a63b80 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638d535d8d0 con 0x5638d55ac940 2023-07-08T12:38:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.065+0000 7eff50a63b80 10 client.0 _ll_register_callbacks cb 0x5638d5477710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-08T12:38:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.065+0000 7eff3dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 0x5638d55c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.065+0000 7eff3dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 0x5638d55c50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49198/0 (socket says 172.21.15.12:49198) 2023-07-08T12:38:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.065+0000 7eff3dd9b700 1 -- 172.21.15.12:0/235658167 learned_addr learned my addr 172.21.15.12:0/235658167 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff3e59c700 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d5589650 0x5638d55aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff3ed9d700 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac940 0x5638d55c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff3dd9b700 1 -- 172.21.15.12:0/235658167 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac940 msgr2=0x5638d55c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac940 0x5638d55c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff3dd9b700 1 -- 172.21.15.12:0/235658167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d5589650 msgr2=0x5638d55aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d5589650 0x5638d55aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff3dd9b700 1 -- 172.21.15.12:0/235658167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff401518e0 con 0x5638d55ac570 2023-07-08T12:38:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff3e59c700 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d5589650 0x5638d55aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 0x5638d55c50e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7eff3005fcb0 tx=0x7eff30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf462556cf79329 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-07-08T12:38:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff1d7fa700 1 -- 172.21.15.12:0/235658167 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3005ff50 con 0x5638d55ac570 2023-07-08T12:38:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff50a63b80 1 -- 172.21.15.12:0/235658167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638d557b0a0 con 0x5638d55ac570 2023-07-08T12:38:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff1d7fa700 1 -- 172.21.15.12:0/235658167 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7eff30053c80 con 0x5638d55ac570 2023-07-08T12:38:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff1d7fa700 1 -- 172.21.15.12:0/235658167 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff30053e40 con 0x5638d55ac570 2023-07-08T12:38:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff50a63b80 10 client.4431 fetch_fsmap learned FSMap version 7 2023-07-08T12:38:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff50a63b80 1 -- 172.21.15.12:0/235658167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5638d52f0c10 con 0x5638d55ac570 2023-07-08T12:38:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff1d7fa700 1 -- 172.21.15.12:0/235658167 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5638d557b0a0 con 0x5638d55ac570 2023-07-08T12:38:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff1d7fa700 1 -- 172.21.15.12:0/235658167 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7eff30076bb0 con 0x5638d55ac570 2023-07-08T12:38:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff50a63b80 10 client.4431 fetch_fsmap finished waiting for FSMap version 7 2023-07-08T12:38:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff50a63b80 10 client.4431 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-08T12:38:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff50a63b80 20 client.4431 populate_metadata read hostname 'smithi012' 2023-07-08T12:38:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.066+0000 7eff50a63b80 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x5638d55b5150 0x5638d55b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.067+0000 7eff50a63b80 4 client.4431 mds_command: new command op to 4290 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-08T12:38:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.067+0000 7eff50a63b80 1 -- 172.21.15.12:0/235658167 --> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638d54d2e60 con 0x5638d55b5150 2023-07-08T12:38:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.067+0000 7eff3e59c700 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x5638d55b5150 0x5638d55b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.067+0000 7eff1d7fa700 10 client.4431 ms_handle_connect on v2:172.21.15.12:6833/3301134992 2023-07-08T12:38:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.067+0000 7eff3e59c700 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x5638d55b5150 0x5638d55b5520 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=645904570ef57553 server_cookie=276cddaa9ae6fe86 in_seq=0 out_seq=0 2023-07-08T12:38:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.067+0000 7eff1d7fa700 1 -- 172.21.15.12:0/235658167 <== mds.0 v2:172.21.15.12:6833/3301134992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5638d54d2e60 con 0x5638d55b5150 2023-07-08T12:38:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.067+0000 7eff1d7fa700 10 client.4431 handle_command_reply: tid=0 2023-07-08T12:38:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.067+0000 7eff50a63b80 1 client.4431 shutdown 2023-07-08T12:38:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.068+0000 7eff50a63b80 1 -- 172.21.15.12:0/235658167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 msgr2=0x5638d55c50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.068+0000 7eff50a63b80 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 0x5638d55c50e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7eff3005fcb0 tx=0x7eff30000bc0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.068+0000 7eff50a63b80 1 -- 172.21.15.12:0/235658167 shutdown_connections 2023-07-08T12:38:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.068+0000 7eff50a63b80 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac940 0x5638d55c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.068+0000 7eff50a63b80 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d5589650 0x5638d55aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.068+0000 7eff50a63b80 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x5638d55b5150 0x5638d55b5520 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.068+0000 7eff50a63b80 1 --2- 172.21.15.12:0/235658167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 0x5638d55c50e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.068+0000 7eff1d7fa700 0 client.4431 ms_handle_reset on v2:172.21.15.12:6833/3301134992 2023-07-08T12:38:16.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.068+0000 7eff50a63b80 1 -- 172.21.15.12:0/235658167 >> 172.21.15.12:0/235658167 conn(0x5638d557da90 msgr2=0x5638d5584ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:16.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.068+0000 7eff50a63b80 1 -- 172.21.15.12:0/235658167 shutdown_connections 2023-07-08T12:38:16.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.068+0000 7eff50a63b80 1 -- 172.21.15.12:0/235658167 wait complete. 2023-07-08T12:38:16.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.068+0000 7eff50a63b80 20 client.4431 trim_cache size 0 max 16384 2023-07-08T12:38:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.086+0000 7eff50a63b80 1 Processor -- start 2023-07-08T12:38:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.086+0000 7eff50a63b80 1 -- start start 2023-07-08T12:38:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.086+0000 7eff50a63b80 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d5589650 0x5638d5582e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.086+0000 7eff50a63b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 0x5638d55b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.086+0000 7eff50a63b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d55ac940 0x5638d55cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.086+0000 7eff50a63b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d55853e0 con 0x5638d55ac570 2023-07-08T12:38:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.086+0000 7eff50a63b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d55aa7f0 con 0x5638d55ac940 2023-07-08T12:38:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.086+0000 7eff50a63b80 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638d558a0a0 con 0x5638d5589650 2023-07-08T12:38:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff3dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 0x5638d55b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff3dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 0x5638d55b6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49206/0 (socket says 172.21.15.12:49206) 2023-07-08T12:38:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff3dd9b700 1 -- 172.21.15.12:0/4145148206 learned_addr learned my addr 172.21.15.12:0/4145148206 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff3e59c700 1 --2- 172.21.15.12:0/4145148206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d5589650 0x5638d5582e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff3ed9d700 1 --2- 172.21.15.12:0/4145148206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d55ac940 0x5638d55cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff3dd9b700 1 -- 172.21.15.12:0/4145148206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d5589650 msgr2=0x5638d5582e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/4145148206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d5589650 0x5638d5582e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff3dd9b700 1 -- 172.21.15.12:0/4145148206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d55ac940 msgr2=0x5638d55cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:16.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/4145148206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d55ac940 0x5638d55cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff3dd9b700 1 -- 172.21.15.12:0/4145148206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638d532e200 con 0x5638d55ac570 2023-07-08T12:38:16.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/4145148206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 0x5638d55b6790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff300578c0 tx=0x7eff3006d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef12c72e375de4a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:16.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff2cff9700 1 -- 172.21.15.12:0/4145148206 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30081400 con 0x5638d55ac570 2023-07-08T12:38:16.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff2cff9700 1 -- 172.21.15.12:0/4145148206 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff30053c80 con 0x5638d55ac570 2023-07-08T12:38:16.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.087+0000 7eff2cff9700 1 -- 172.21.15.12:0/4145148206 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3005ff70 con 0x5638d55ac570 2023-07-08T12:38:16.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.088+0000 7eff50a63b80 1 -- 172.21.15.12:0/4145148206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 msgr2=0x5638d55b6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.088+0000 7eff50a63b80 1 --2- 172.21.15.12:0/4145148206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 0x5638d55b6790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff300578c0 tx=0x7eff3006d070 comp rx=0 tx=0).stop 2023-07-08T12:38:16.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.088+0000 7eff50a63b80 1 -- 172.21.15.12:0/4145148206 shutdown_connections 2023-07-08T12:38:16.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.088+0000 7eff50a63b80 1 --2- 172.21.15.12:0/4145148206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d55ac940 0x5638d55cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.088+0000 7eff50a63b80 1 --2- 172.21.15.12:0/4145148206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d5589650 0x5638d5582e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.088+0000 7eff50a63b80 1 --2- 172.21.15.12:0/4145148206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac570 0x5638d55b6790 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.088+0000 7eff50a63b80 1 -- 172.21.15.12:0/4145148206 >> 172.21.15.12:0/4145148206 conn(0x5638d557da90 msgr2=0x5638d557f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:16.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.088+0000 7eff50a63b80 1 -- 172.21.15.12:0/4145148206 shutdown_connections 2023-07-08T12:38:16.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.088+0000 7eff50a63b80 1 -- 172.21.15.12:0/4145148206 wait complete. 2023-07-08T12:38:16.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.088+0000 7eff50a63b80 1 -- start start 2023-07-08T12:38:16.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.088+0000 7eff50a63b80 1 Processor -- start 2023-07-08T12:38:16.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff50a63b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d5589650 0x5638d55c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff50a63b80 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac570 0x5638d55c6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff50a63b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac940 0x5638d55c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff50a63b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d55b6c90 con 0x5638d55ac940 2023-07-08T12:38:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff50a63b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d5581d20 con 0x5638d5589650 2023-07-08T12:38:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff50a63b80 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638d5638d50 con 0x5638d55ac570 2023-07-08T12:38:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff50a63b80 10 client.0 _ll_register_callbacks cb 0x7eff340090e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-08T12:38:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff3ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac940 0x5638d55c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff3ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac940 0x5638d55c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49214/0 (socket says 172.21.15.12:49214) 2023-07-08T12:38:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff3ed9d700 1 -- 172.21.15.12:0/1494544231 learned_addr learned my addr 172.21.15.12:0/1494544231 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:16.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff3e59c700 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d5589650 0x5638d55c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac570 0x5638d55c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff3ed9d700 1 -- 172.21.15.12:0/1494544231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac570 msgr2=0x5638d55c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:16.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff3ed9d700 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac570 0x5638d55c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff3ed9d700 1 -- 172.21.15.12:0/1494544231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d5589650 msgr2=0x5638d55c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:16.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff3ed9d700 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d5589650 0x5638d55c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff3ed9d700 1 -- 172.21.15.12:0/1494544231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff401518e0 con 0x5638d55ac940 2023-07-08T12:38:16.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff3dd9b700 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac570 0x5638d55c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:16.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff3ed9d700 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac940 0x5638d55c6840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7eff4015dd70 tx=0x7eff380125a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8c0d8aab6bd32e server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:16.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-07-08T12:38:16.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.090+0000 7eff1d7fa700 1 -- 172.21.15.12:0/1494544231 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38012b00 con 0x5638d55ac940 2023-07-08T12:38:16.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff1d7fa700 1 -- 172.21.15.12:0/1494544231 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7eff38012c60 con 0x5638d55ac940 2023-07-08T12:38:16.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff50a63b80 1 -- 172.21.15.12:0/1494544231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff30057380 con 0x5638d55ac940 2023-07-08T12:38:16.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff1d7fa700 1 -- 172.21.15.12:0/1494544231 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff38012e40 con 0x5638d55ac940 2023-07-08T12:38:16.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff50a63b80 10 client.4437 fetch_fsmap learned FSMap version 7 2023-07-08T12:38:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff50a63b80 1 -- 172.21.15.12:0/1494544231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5638d5306260 con 0x5638d55ac940 2023-07-08T12:38:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff1d7fa700 1 -- 172.21.15.12:0/1494544231 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff30057380 con 0x5638d55ac940 2023-07-08T12:38:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff1d7fa700 1 -- 172.21.15.12:0/1494544231 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7eff38003a00 con 0x5638d55ac940 2023-07-08T12:38:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff50a63b80 10 client.4437 fetch_fsmap finished waiting for FSMap version 7 2023-07-08T12:38:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff50a63b80 10 client.4437 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-08T12:38:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff50a63b80 20 client.4437 populate_metadata read hostname 'smithi012' 2023-07-08T12:38:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff50a63b80 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x5638d563bd20 0x5638d563c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff50a63b80 4 client.4437 mds_command: new command op to 4290 tid=0[{"prefix": "status"}] 2023-07-08T12:38:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff50a63b80 1 -- 172.21.15.12:0/1494544231 --> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5638d5495000 con 0x5638d563bd20 2023-07-08T12:38:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff3e59c700 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x5638d563bd20 0x5638d563c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff1d7fa700 10 client.4437 ms_handle_connect on v2:172.21.15.12:6833/3301134992 2023-07-08T12:38:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.091+0000 7eff3e59c700 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x5638d563bd20 0x5638d563c0f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed51758948b4da2a server_cookie=7399d97f24c69a88 in_seq=0 out_seq=0 2023-07-08T12:38:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff1d7fa700 1 -- 172.21.15.12:0/1494544231 <== mds.0 v2:172.21.15.12:6833/3301134992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5638d5495000 con 0x5638d563bd20 2023-07-08T12:38:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff1d7fa700 10 client.4437 handle_command_reply: tid=0 2023-07-08T12:38:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff50a63b80 1 client.4437 shutdown 2023-07-08T12:38:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff50a63b80 1 -- 172.21.15.12:0/1494544231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac940 msgr2=0x5638d55c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff50a63b80 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac940 0x5638d55c6840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7eff4015dd70 tx=0x7eff380125a0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff50a63b80 1 -- 172.21.15.12:0/1494544231 shutdown_connections 2023-07-08T12:38:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff50a63b80 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x5638d563bd20 0x5638d563c0f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff50a63b80 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638d55ac940 0x5638d55c6840 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff1d7fa700 0 client.4437 ms_handle_reset on v2:172.21.15.12:6833/3301134992 2023-07-08T12:38:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff50a63b80 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5638d5589650 0x5638d55c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff50a63b80 1 --2- 172.21.15.12:0/1494544231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5638d55ac570 0x5638d55c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff50a63b80 1 -- 172.21.15.12:0/1494544231 >> 172.21.15.12:0/1494544231 conn(0x5638d557da90 msgr2=0x5638d557f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:16.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff50a63b80 1 -- 172.21.15.12:0/1494544231 shutdown_connections 2023-07-08T12:38:16.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff50a63b80 1 -- 172.21.15.12:0/1494544231 wait complete. 2023-07-08T12:38:16.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff50a63b80 20 client.4437 trim_cache size 0 max 16384 2023-07-08T12:38:16.114 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-07-08T12:38:16.115 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "71d270f5-58a7-4223-8d07-37436121729a", 2023-07-08T12:38:16.115 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2023-07-08T12:38:16.115 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 4290, 2023-07-08T12:38:16.115 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2023-07-08T12:38:16.115 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2023-07-08T12:38:16.115 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2023-07-08T12:38:16.115 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 7.5481880029999999, 2023-07-08T12:38:16.115 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 7, 2023-07-08T12:38:16.116 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 28, 2023-07-08T12:38:16.116 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 28, 2023-07-08T12:38:16.116 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 13.601812677 2023-07-08T12:38:16.116 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-07-08T12:38:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff448de700 1 -- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7eff2006ace0 msgr2=0x7eff2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff448de700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7eff2006ace0 0x7eff2006d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff340090b0 tx=0x7eff34008030 comp rx=0 tx=0).stop 2023-07-08T12:38:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff448de700 1 -- 172.21.15.12:0/4070850868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff40151be0 msgr2=0x7eff401513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff448de700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff40151be0 0x7eff401513a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7eff3000e870 tx=0x7eff3000eb80 comp rx=0 tx=0).stop 2023-07-08T12:38:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff448de700 1 -- 172.21.15.12:0/4070850868 shutdown_connections 2023-07-08T12:38:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff448de700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7eff2006ace0 0x7eff2006d190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff448de700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff401e5030 0x7eff400b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff448de700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff28003be0 0x7eff400c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.092+0000 7eff448de700 1 --2- 172.21.15.12:0/4070850868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff40151be0 0x7eff401513a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.093+0000 7eff448de700 1 -- 172.21.15.12:0/4070850868 >> 172.21.15.12:0/4070850868 conn(0x7eff400cc290 msgr2=0x7eff40155270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.093+0000 7eff448de700 1 -- 172.21.15.12:0/4070850868 shutdown_connections 2023-07-08T12:38:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.093+0000 7eff448de700 1 -- 172.21.15.12:0/4070850868 wait complete. 2023-07-08T12:38:16.119 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-08T12:38:16.120 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-08T12:38:16.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-08T12:38:16.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.243+0000 7f4a4c92d700 1 Processor -- start 2023-07-08T12:38:16.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.243+0000 7f4a4c92d700 1 -- start start 2023-07-08T12:38:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.244+0000 7f4a4c92d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a480b1c70 0x7f4a480ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.244+0000 7f4a4c92d700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4815d230 con 0x7f4a4815d6c0 2023-07-08T12:38:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.244+0000 7f4a4c92d700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a480ceac0 con 0x7f4a480ce1b0 2023-07-08T12:38:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.244+0000 7f4a4c92d700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a480c9e60 con 0x7f4a480b1c70 2023-07-08T12:38:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.244+0000 7f4a45d9b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a480b1c70 0x7f4a480ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.244+0000 7f4a46d9d700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4a4815d6c0 0x7f4a48071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36616/0 (socket says 172.21.15.12:36616) 2023-07-08T12:38:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.244+0000 7f4a45d9b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a480b1c70 0x7f4a480ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49220/0 (socket says 172.21.15.12:49220) 2023-07-08T12:38:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.244+0000 7f4a46d9d700 1 -- 172.21.15.12:0/4157308480 learned_addr learned my addr 172.21.15.12:0/4157308480 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.244+0000 7f4a45d9b700 1 -- 172.21.15.12:0/4157308480 >> v1:172.21.15.160:6789/0 conn(0x7f4a480ce1b0 legacy=0x7f4a480cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.244+0000 7f4a45d9b700 1 -- 172.21.15.12:0/4157308480 >> v1:172.21.15.12:6789/0 conn(0x7f4a4815d6c0 legacy=0x7f4a48071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.244+0000 7f4a4559a700 1 -- 172.21.15.12:0/4157308480 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2117377844 0 0) 0x7f4a4815d230 con 0x7f4a4815d6c0 2023-07-08T12:38:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.244+0000 7f4a45d9b700 1 -- 172.21.15.12:0/4157308480 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480cdbc0 con 0x7f4a480b1c70 2023-07-08T12:38:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.244+0000 7f4a45d9b700 1 --2- 172.21.15.12:0/4157308480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a480b1c70 0x7f4a480ce580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009840 tx=0x7f4a3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9674faed5a60ee5 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.245+0000 7f4a4559a700 1 -- 172.21.15.12:0/4157308480 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c011830 con 0x7f4a480b1c70 2023-07-08T12:38:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.245+0000 7f4a4559a700 1 -- 172.21.15.12:0/4157308480 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c011990 con 0x7f4a480b1c70 2023-07-08T12:38:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.245+0000 7f4a4559a700 1 -- 172.21.15.12:0/4157308480 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c011d20 con 0x7f4a480b1c70 2023-07-08T12:38:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.245+0000 7f4a4c92d700 1 -- 172.21.15.12:0/4157308480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a480b1c70 msgr2=0x7f4a480ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.245+0000 7f4a4c92d700 1 --2- 172.21.15.12:0/4157308480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a480b1c70 0x7f4a480ce580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009840 tx=0x7f4a3c009b50 comp rx=0 tx=0).stop 2023-07-08T12:38:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.245+0000 7f4a4c92d700 1 -- 172.21.15.12:0/4157308480 shutdown_connections 2023-07-08T12:38:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.245+0000 7f4a4c92d700 1 --2- 172.21.15.12:0/4157308480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a480b1c70 0x7f4a480ce580 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.245+0000 7f4a4c92d700 1 -- 172.21.15.12:0/4157308480 >> 172.21.15.12:0/4157308480 conn(0x7f4a480cc290 msgr2=0x7f4a480cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.245+0000 7f4a4c92d700 1 -- 172.21.15.12:0/4157308480 shutdown_connections 2023-07-08T12:38:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.245+0000 7f4a4c92d700 1 -- 172.21.15.12:0/4157308480 wait complete. 2023-07-08T12:38:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.247+0000 7f4a4c92d700 1 Processor -- start 2023-07-08T12:38:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.247+0000 7f4a4c92d700 1 -- start start 2023-07-08T12:38:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4c92d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a480b1c70 0x7f4a480b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4c92d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a480ce1b0 0x7f4a480c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4c92d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a4815d6c0 0x7f4a480cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4c92d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480b4100 con 0x7f4a480b1c70 2023-07-08T12:38:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4c92d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480b43e0 con 0x7f4a480ce1b0 2023-07-08T12:38:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4c92d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480b46c0 con 0x7f4a4815d6c0 2023-07-08T12:38:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a480b1c70 0x7f4a480b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a480b1c70 0x7f4a480b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49224/0 (socket says 172.21.15.12:49224) 2023-07-08T12:38:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4659c700 1 -- 172.21.15.12:0/392743215 learned_addr learned my addr 172.21.15.12:0/392743215 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a45d9b700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a480ce1b0 0x7f4a480c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a46d9d700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a4815d6c0 0x7f4a480cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4659c700 1 -- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a4815d6c0 msgr2=0x7f4a480cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4659c700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a4815d6c0 0x7f4a480cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4659c700 1 -- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a480ce1b0 msgr2=0x7f4a480c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4659c700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a480ce1b0 0x7f4a480c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.248+0000 7f4a4659c700 1 -- 172.21.15.12:0/392743215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c0094a0 con 0x7f4a480b1c70 2023-07-08T12:38:16.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.249+0000 7f4a4659c700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a480b1c70 0x7f4a480b51a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c450 tx=0x7f4a3800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:16.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.249+0000 7f4a36ffd700 1 -- 172.21.15.12:0/392743215 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380052d0 con 0x7f4a480b1c70 2023-07-08T12:38:16.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.249+0000 7f4a4c92d700 1 -- 172.21.15.12:0/392743215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a480b49a0 con 0x7f4a480b1c70 2023-07-08T12:38:16.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.249+0000 7f4a36ffd700 1 -- 172.21.15.12:0/392743215 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a38005430 con 0x7f4a480b1c70 2023-07-08T12:38:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.249+0000 7f4a36ffd700 1 -- 172.21.15.12:0/392743215 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380038c0 con 0x7f4a480b1c70 2023-07-08T12:38:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.249+0000 7f4a4c92d700 1 -- 172.21.15.12:0/392743215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a480c81b0 con 0x7f4a480b1c70 2023-07-08T12:38:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.250+0000 7f4a4c92d700 1 -- 172.21.15.12:0/392743215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a480b4ec0 con 0x7f4a480b1c70 2023-07-08T12:38:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.250+0000 7f4a36ffd700 1 -- 172.21.15.12:0/392743215 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a38003a20 con 0x7f4a480b1c70 2023-07-08T12:38:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.251+0000 7f4a36ffd700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a2806ace0 0x7f4a2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.251+0000 7f4a36ffd700 1 -- 172.21.15.12:0/392743215 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4a38088aa0 con 0x7f4a480b1c70 2023-07-08T12:38:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.251+0000 7f4a45d9b700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a2806ace0 0x7f4a2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.251+0000 7f4a45d9b700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a2806ace0 0x7f4a2806d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4a48071a20 tx=0x7f4a3c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:16.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.255+0000 7f4a36ffd700 1 -- 172.21.15.12:0/392743215 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4a480b4ec0 con 0x7f4a480b1c70 2023-07-08T12:38:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.503+0000 7f4a4c92d700 1 -- 172.21.15.12:0/392743215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4a480b4ec0 con 0x7f4a480b1c70 2023-07-08T12:38:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.503+0000 7f4a36ffd700 1 -- 172.21.15.12:0/392743215 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4a38053870 con 0x7f4a480b1c70 2023-07-08T12:38:16.506 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-08T12:38:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.505+0000 7f4a4c92d700 1 -- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a2806ace0 msgr2=0x7f4a2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.505+0000 7f4a4c92d700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a2806ace0 0x7f4a2806d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4a48071a20 tx=0x7f4a3c01c040 comp rx=0 tx=0).stop 2023-07-08T12:38:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.505+0000 7f4a4c92d700 1 -- 172.21.15.12:0/392743215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a480b1c70 msgr2=0x7f4a480b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.505+0000 7f4a4c92d700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a480b1c70 0x7f4a480b51a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c450 tx=0x7f4a3800c810 comp rx=0 tx=0).stop 2023-07-08T12:38:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.506+0000 7f4a4c92d700 1 -- 172.21.15.12:0/392743215 shutdown_connections 2023-07-08T12:38:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.506+0000 7f4a4c92d700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a2806ace0 0x7f4a2806d190 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.506+0000 7f4a4c92d700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a4815d6c0 0x7f4a480cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.506+0000 7f4a4c92d700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a480b1c70 0x7f4a480b51a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.506+0000 7f4a4c92d700 1 --2- 172.21.15.12:0/392743215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a480ce1b0 0x7f4a480c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.506+0000 7f4a4c92d700 1 -- 172.21.15.12:0/392743215 >> 172.21.15.12:0/392743215 conn(0x7f4a480cc290 msgr2=0x7f4a48071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.506+0000 7f4a4c92d700 1 -- 172.21.15.12:0/392743215 shutdown_connections 2023-07-08T12:38:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:16.506+0000 7f4a4c92d700 1 -- 172.21.15.12:0/392743215 wait complete. 2023-07-08T12:38:16.520 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-08T12:38:16.520 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-08T12:38:16.520 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2023-07-08T12:38:16.521 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-07-08T12:38:16.521 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-08T12:38:16.521 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-08T12:38:16.521 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-08T12:38:16.522 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-08T12:38:16.522 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-07-08T12:38:16.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:16.523 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2023-07-08T12:38:16.540 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-08T12:38:16.540 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-08T12:38:16.541 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2023-07-08T12:38:16.541 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:16.541 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2023-07-08T12:38:16.541 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:16.541 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:16.541 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:16.541 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:16.541 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:16.542 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-08T12:38:16.542 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:16.542 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-08T12:38:16.543 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42153sec preferred_lft 42153sec 2023-07-08T12:38:16.543 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2023-07-08T12:38:16.543 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:16.543 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:16.544 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:16.545 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-08T12:38:16.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:16.547 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:16.547 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:16.548 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2023-07-08T12:38:16.549 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2023-07-08T12:38:16.549 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2023-07-08T12:38:16.549 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-08T12:38:16.549 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:16.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:16.730 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-08T12:38:16.759 INFO:teuthology.orchestra.run.smithi012.stdout:1 2023-07-08T12:38:16.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:16.760 DEBUG:teuthology.orchestra.run.smithi012:> route 2023-07-08T12:38:16.835 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2023-07-08T12:38:16.835 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-08T12:38:16.836 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-08T12:38:16.836 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-08T12:38:16.836 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-08T12:38:16.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:16.837 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:16.838 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:16.839 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-08T12:38:16.839 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-08T12:38:16.840 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-08T12:38:16.841 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:17.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:17.102 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2023-07-08T12:38:17.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:17.121 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2023-07-08T12:38:17.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:17.178 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:17.179 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:17.179 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-08T12:38:17.180 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-07-08T12:38:17.180 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:17.273 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-07-08T12:38:17.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:17.274 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:17.274 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:17.274 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-07-08T12:38:17.275 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-08T12:38:17.275 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-07-08T12:38:17.275 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-07-08T12:38:17.276 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-07-08T12:38:17.276 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:17.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:17.492 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:17.492 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:17.493 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2023-07-08T12:38:17.493 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2023-07-08T12:38:17.493 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:17.595 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-07-08T12:38:17.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-08T12:38:17.596 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-07-08T12:38:17.619 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-07-08T12:38:17.619 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-08T12:38:17.619 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-07-08T12:38:17.676 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2023-07-08T12:38:17.744 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-08T12:38:17.744 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-08T12:38:17.812 INFO:teuthology.orchestra.run.smithi012.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-08T12:38:17.813 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-08T12:38:17.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:17.815 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2023-07-08T12:38:17.872 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-08T12:38:17.873 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-07-08T12:38:17.914 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2023-07-08T12:38:17.941 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-08T12:38:17.942 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-08T12:38:18.008 INFO:teuthology.orchestra.run.smithi012.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-08T12:38:18.009 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-08T12:38:18.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:18.010 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2023-07-08T12:38:18.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:2023-07-08T12:38:18.019+0000 7f48b764e5c0 -1 init, newargv = 0x55d3eccf28b0 newargc=15 2023-07-08T12:38:18.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[138846]: starting ceph client 2023-07-08T12:38:18.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[138846]: starting fuse 2023-07-08T12:38:18.039 INFO:teuthology.orchestra.run.smithi012.stdout:47 2023-07-08T12:38:18.039 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-07-08T12:38:18.040 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-08T12:38:18.040 DEBUG:teuthology.orchestra.run.smithi012:> import glob 2023-07-08T12:38:18.040 DEBUG:teuthology.orchestra.run.smithi012:> import re 2023-07-08T12:38:18.041 DEBUG:teuthology.orchestra.run.smithi012:> import os 2023-07-08T12:38:18.041 DEBUG:teuthology.orchestra.run.smithi012:> import subprocess 2023-07-08T12:38:18.041 DEBUG:teuthology.orchestra.run.smithi012:> 2023-07-08T12:38:18.041 DEBUG:teuthology.orchestra.run.smithi012:> def _find_admin_socket(client_name): 2023-07-08T12:38:18.042 DEBUG:teuthology.orchestra.run.smithi012:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-07-08T12:38:18.042 DEBUG:teuthology.orchestra.run.smithi012:> files = glob.glob(asok_path) 2023-07-08T12:38:18.042 DEBUG:teuthology.orchestra.run.smithi012:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-07-08T12:38:18.042 DEBUG:teuthology.orchestra.run.smithi012:> 2023-07-08T12:38:18.043 DEBUG:teuthology.orchestra.run.smithi012:> # Given a non-glob path, it better be there 2023-07-08T12:38:18.043 DEBUG:teuthology.orchestra.run.smithi012:> if "*" not in asok_path: 2023-07-08T12:38:18.043 DEBUG:teuthology.orchestra.run.smithi012:> assert(len(files) == 1) 2023-07-08T12:38:18.044 DEBUG:teuthology.orchestra.run.smithi012:> return files[0] 2023-07-08T12:38:18.044 DEBUG:teuthology.orchestra.run.smithi012:> 2023-07-08T12:38:18.044 DEBUG:teuthology.orchestra.run.smithi012:> for f in files: 2023-07-08T12:38:18.044 DEBUG:teuthology.orchestra.run.smithi012:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-08T12:38:18.045 DEBUG:teuthology.orchestra.run.smithi012:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-08T12:38:18.045 DEBUG:teuthology.orchestra.run.smithi012:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-08T12:38:18.045 DEBUG:teuthology.orchestra.run.smithi012:> contents = proc_f.read() 2023-07-08T12:38:18.045 DEBUG:teuthology.orchestra.run.smithi012:> if mountpoint in contents: 2023-07-08T12:38:18.046 DEBUG:teuthology.orchestra.run.smithi012:> return f 2023-07-08T12:38:18.046 DEBUG:teuthology.orchestra.run.smithi012:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-08T12:38:18.046 DEBUG:teuthology.orchestra.run.smithi012:> 2023-07-08T12:38:18.047 DEBUG:teuthology.orchestra.run.smithi012:> print(_find_admin_socket("client.admin")) 2023-07-08T12:38:18.047 DEBUG:teuthology.orchestra.run.smithi012:> ' 2023-07-08T12:38:18.164 INFO:teuthology.orchestra.run.smithi012.stdout:/var/run/ceph/ceph-client.admin.138846.asok 2023-07-08T12:38:18.345 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.138846.asok 2023-07-08T12:38:18.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:18.346 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.138846.asok status 2023-07-08T12:38:18.510 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-07-08T12:38:18.510 INFO:teuthology.orchestra.run.smithi012.stdout: "metadata": { 2023-07-08T12:38:18.510 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_sha1": "f3a06ccd0ae55e9794bc8ce562926656b2738063", 2023-07-08T12:38:18.510 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 18.0.0-4738-gf3a06ccd (f3a06ccd0ae55e9794bc8ce562926656b2738063) reef (dev)", 2023-07-08T12:38:18.511 INFO:teuthology.orchestra.run.smithi012.stdout: "entity_id": "admin", 2023-07-08T12:38:18.511 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": "smithi012", 2023-07-08T12:38:18.511 INFO:teuthology.orchestra.run.smithi012.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-07-08T12:38:18.511 INFO:teuthology.orchestra.run.smithi012.stdout: "pid": "138846", 2023-07-08T12:38:18.511 INFO:teuthology.orchestra.run.smithi012.stdout: "root": "/" 2023-07-08T12:38:18.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:38:18.511 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_count": 0, 2023-07-08T12:38:18.512 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_pinned_count": 0, 2023-07-08T12:38:18.512 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 4452, 2023-07-08T12:38:18.512 INFO:teuthology.orchestra.run.smithi012.stdout: "inst": { 2023-07-08T12:38:18.512 INFO:teuthology.orchestra.run.smithi012.stdout: "name": { 2023-07-08T12:38:18.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "client", 2023-07-08T12:38:18.512 INFO:teuthology.orchestra.run.smithi012.stdout: "num": 4452 2023-07-08T12:38:18.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:38:18.513 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2023-07-08T12:38:18.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2023-07-08T12:38:18.513 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2023-07-08T12:38:18.513 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3447540059 2023-07-08T12:38:18.513 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:38:18.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:38:18.513 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2023-07-08T12:38:18.514 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2023-07-08T12:38:18.514 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2023-07-08T12:38:18.514 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3447540059 2023-07-08T12:38:18.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:38:18.514 INFO:teuthology.orchestra.run.smithi012.stdout: "inst_str": "client.4452 192.168.0.1:0/3447540059", 2023-07-08T12:38:18.514 INFO:teuthology.orchestra.run.smithi012.stdout: "addr_str": "192.168.0.1:0/3447540059", 2023-07-08T12:38:18.514 INFO:teuthology.orchestra.run.smithi012.stdout: "inode_count": 1, 2023-07-08T12:38:18.515 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_epoch": 7, 2023-07-08T12:38:18.515 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch": 28, 2023-07-08T12:38:18.515 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch_barrier": 0, 2023-07-08T12:38:18.515 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklisted": false, 2023-07-08T12:38:18.515 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs" 2023-07-08T12:38:18.515 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-07-08T12:38:18.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:18.521 DEBUG:teuthology.orchestra.run.smithi012:> stat --file-system '--printf=%T 2023-07-08T12:38:18.521 DEBUG:teuthology.orchestra.run.smithi012:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-07-08T12:38:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:fuseblk 2023-07-08T12:38:18.539 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-07-08T12:38:18.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:18.540 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-07-08T12:38:18.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:18.608 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-07-08T12:38:18.751 INFO:teuthology.orchestra.run.smithi012.stdout:# file: . 2023-07-08T12:38:18.751 INFO:teuthology.orchestra.run.smithi012.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-07-08T12:38:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:18.751 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:38:18.752 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:38:18.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:18.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.932+0000 7f7d25745700 1 Processor -- start 2023-07-08T12:38:18.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.932+0000 7f7d25745700 1 -- start start 2023-07-08T12:38:18.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.933+0000 7f7d25745700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7d2006b470 0x7f7d20070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:18.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.933+0000 7f7d25745700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d200cd070 con 0x7f7d20069f30 2023-07-08T12:38:18.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.933+0000 7f7d25745700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d200ccab0 con 0x7f7d201607f0 2023-07-08T12:38:18.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.933+0000 7f7d25745700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d200ce690 con 0x7f7d2006b470 2023-07-08T12:38:18.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.933+0000 7f7d1effd700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f7d20069f30 0x7f7d2006a300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36646/0 (socket says 172.21.15.12:36646) 2023-07-08T12:38:18.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.933+0000 7f7d1effd700 1 -- 172.21.15.12:0/3968239680 learned_addr learned my addr 172.21.15.12:0/3968239680 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:18.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.933+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610729825 0 0) 0x7f7d200cd070 con 0x7f7d20069f30 2023-07-08T12:38:18.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.933+0000 7f7d1e7fc700 1 --2- 172.21.15.12:0/3968239680 >> v2:172.21.15.160:3300/0 conn(0x7f7d2006b470 0x7f7d20070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:18.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.933+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d080035b0 con 0x7f7d20069f30 2023-07-08T12:38:18.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2196975089 0 0) 0x7f7d080035b0 con 0x7f7d20069f30 2023-07-08T12:38:18.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 >> v2:172.21.15.160:3300/0 conn(0x7f7d2006b470 msgr2=0x7f7d20070bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:18.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1dffb700 1 --2- 172.21.15.12:0/3968239680 >> v2:172.21.15.160:3300/0 conn(0x7f7d2006b470 0x7f7d20070bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:18.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 >> v1:172.21.15.160:6789/0 conn(0x7f7d201607f0 legacy=0x7f7d2015c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:18.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20160c50 con 0x7f7d20069f30 2023-07-08T12:38:18.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f7d14001670 con 0x7f7d20069f30 2023-07-08T12:38:18.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 >> v1:172.21.15.12:6789/0 conn(0x7f7d20069f30 legacy=0x7f7d2006a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:18.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1dffb700 1 --2- 172.21.15.12:0/3968239680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d08003a00 0x7f7d08003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:18.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1dffb700 1 --2- 172.21.15.12:0/3968239680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d08004850 0x7f7d08004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:18.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1dffb700 1 --2- 172.21.15.12:0/3968239680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d08005200 0x7f7d0800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:18.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d200ce690 con 0x7f7d08003a00 2023-07-08T12:38:18.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0800c080 con 0x7f7d08004850 2023-07-08T12:38:18.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0800c240 con 0x7f7d08005200 2023-07-08T12:38:18.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1effd700 1 --2- 172.21.15.12:0/3968239680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d08003a00 0x7f7d08003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:18.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1e7fc700 1 --2- 172.21.15.12:0/3968239680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d08004850 0x7f7d08004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:18.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1f7fe700 1 --2- 172.21.15.12:0/3968239680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d08005200 0x7f7d0800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:18.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1effd700 1 -- 172.21.15.12:0/3968239680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d08005200 msgr2=0x7f7d0800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:18.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1effd700 1 --2- 172.21.15.12:0/3968239680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d08005200 0x7f7d0800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:18.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1effd700 1 -- 172.21.15.12:0/3968239680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d08004850 msgr2=0x7f7d08004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:18.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1effd700 1 --2- 172.21.15.12:0/3968239680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d08004850 0x7f7d08004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:18.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1f7fe700 1 -- 172.21.15.12:0/3968239680 reap_dead start 2023-07-08T12:38:18.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1effd700 1 -- 172.21.15.12:0/3968239680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d080035b0 con 0x7f7d08003a00 2023-07-08T12:38:18.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1e7fc700 1 --2- 172.21.15.12:0/3968239680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d08004850 0x7f7d08004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:38:18.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.934+0000 7f7d1effd700 1 --2- 172.21.15.12:0/3968239680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d08003a00 0x7f7d08003e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400c2a0 tx=0x7f7d08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc92cce0e1a621 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:18.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.935+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14007c10 con 0x7f7d08003a00 2023-07-08T12:38:18.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.935+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1400cda0 con 0x7f7d08003a00 2023-07-08T12:38:18.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.935+0000 7f7d1dffb700 1 -- 172.21.15.12:0/3968239680 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14005ba0 con 0x7f7d08003a00 2023-07-08T12:38:18.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.935+0000 7f7d25745700 1 -- 172.21.15.12:0/3968239680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d08003a00 msgr2=0x7f7d08003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:18.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.935+0000 7f7d25745700 1 --2- 172.21.15.12:0/3968239680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d08003a00 0x7f7d08003e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400c2a0 tx=0x7f7d08005880 comp rx=0 tx=0).stop 2023-07-08T12:38:18.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.935+0000 7f7d25745700 1 -- 172.21.15.12:0/3968239680 shutdown_connections 2023-07-08T12:38:18.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.935+0000 7f7d25745700 1 --2- 172.21.15.12:0/3968239680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d08003a00 0x7f7d08003e10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:18.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.935+0000 7f7d25745700 1 -- 172.21.15.12:0/3968239680 >> 172.21.15.12:0/3968239680 conn(0x7f7d200cb9f0 msgr2=0x7f7d200cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:18.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.935+0000 7f7d25745700 1 -- 172.21.15.12:0/3968239680 shutdown_connections 2023-07-08T12:38:18.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.935+0000 7f7d25745700 1 -- 172.21.15.12:0/3968239680 wait complete. 2023-07-08T12:38:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.937+0000 7f7d25745700 1 Processor -- start 2023-07-08T12:38:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.937+0000 7f7d25745700 1 -- start start 2023-07-08T12:38:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.937+0000 7f7d25745700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d08003a00 0x7f7d2015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.937+0000 7f7d25745700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d20156b70 0x7f7d20146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.937+0000 7f7d25745700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20154eb0 0x7f7d20155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.937+0000 7f7d25745700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2015da20 con 0x7f7d08003a00 2023-07-08T12:38:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.937+0000 7f7d25745700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d20154a20 con 0x7f7d20154eb0 2023-07-08T12:38:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.937+0000 7f7d25745700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d20154160 con 0x7f7d20156b70 2023-07-08T12:38:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.937+0000 7f7d1effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d08003a00 0x7f7d2015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.937+0000 7f7d1effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d08003a00 0x7f7d2015d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49260/0 (socket says 172.21.15.12:49260) 2023-07-08T12:38:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.937+0000 7f7d1effd700 1 -- 172.21.15.12:0/1999509704 learned_addr learned my addr 172.21.15.12:0/1999509704 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.938+0000 7f7d1effd700 1 -- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d20156b70 msgr2=0x7f7d20146380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.938+0000 7f7d1e7fc700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d20156b70 0x7f7d20146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.938+0000 7f7d1f7fe700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20154eb0 0x7f7d20155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.938+0000 7f7d1effd700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d20156b70 0x7f7d20146380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.938+0000 7f7d1effd700 1 -- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20154eb0 msgr2=0x7f7d20155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.938+0000 7f7d1effd700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20154eb0 0x7f7d20155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.938+0000 7f7d1effd700 1 -- 172.21.15.12:0/1999509704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d201543d0 con 0x7f7d08003a00 2023-07-08T12:38:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.938+0000 7f7d1effd700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d08003a00 0x7f7d2015d4e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d140011d0 tx=0x7f7d2015c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.938+0000 7f7d077fe700 1 -- 172.21.15.12:0/1999509704 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14001630 con 0x7f7d08003a00 2023-07-08T12:38:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.938+0000 7f7d077fe700 1 -- 172.21.15.12:0/1999509704 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d140070d0 con 0x7f7d08003a00 2023-07-08T12:38:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.938+0000 7f7d25745700 1 -- 172.21.15.12:0/1999509704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d201546b0 con 0x7f7d08003a00 2023-07-08T12:38:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.938+0000 7f7d077fe700 1 -- 172.21.15.12:0/1999509704 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14007480 con 0x7f7d08003a00 2023-07-08T12:38:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.938+0000 7f7d25745700 1 -- 172.21.15.12:0/1999509704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d20157370 con 0x7f7d08003a00 2023-07-08T12:38:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.939+0000 7f7d25745700 1 -- 172.21.15.12:0/1999509704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2004ea60 con 0x7f7d08003a00 2023-07-08T12:38:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.939+0000 7f7d077fe700 1 -- 172.21.15.12:0/1999509704 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d140075e0 con 0x7f7d08003a00 2023-07-08T12:38:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.940+0000 7f7d077fe700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7d0c06ace0 0x7f7d0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.940+0000 7f7d077fe700 1 -- 172.21.15.12:0/1999509704 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7d1408af50 con 0x7f7d08003a00 2023-07-08T12:38:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.940+0000 7f7d1e7fc700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7d0c06ace0 0x7f7d0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.941+0000 7f7d1e7fc700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7d0c06ace0 0x7f7d0c06d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d201f5790 tx=0x7f7d10008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:18.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:18.944+0000 7f7d077fe700 1 -- 172.21.15.12:0/1999509704 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7d14055e60 con 0x7f7d08003a00 2023-07-08T12:38:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.142+0000 7f7d25745700 1 -- 172.21.15.12:0/1999509704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d20065f50 con 0x7f7d08003a00 2023-07-08T12:38:19.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.143+0000 7f7d077fe700 1 -- 172.21.15.12:0/1999509704 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f7d14059480 con 0x7f7d08003a00 2023-07-08T12:38:19.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:19.146 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:12.993803+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.145+0000 7f7d25745700 1 -- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7d0c06ace0 msgr2=0x7f7d0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.145+0000 7f7d25745700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7d0c06ace0 0x7f7d0c06d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d201f5790 tx=0x7f7d10008040 comp rx=0 tx=0).stop 2023-07-08T12:38:19.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.145+0000 7f7d25745700 1 -- 172.21.15.12:0/1999509704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d08003a00 msgr2=0x7f7d2015d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.145+0000 7f7d25745700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d08003a00 0x7f7d2015d4e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d140011d0 tx=0x7f7d2015c7f0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.146+0000 7f7d25745700 1 -- 172.21.15.12:0/1999509704 shutdown_connections 2023-07-08T12:38:19.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.146+0000 7f7d25745700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7d0c06ace0 0x7f7d0c06d190 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.146+0000 7f7d25745700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d20154eb0 0x7f7d20155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.146+0000 7f7d25745700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d08003a00 0x7f7d2015d4e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.146+0000 7f7d25745700 1 --2- 172.21.15.12:0/1999509704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d20156b70 0x7f7d20146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.146+0000 7f7d25745700 1 -- 172.21.15.12:0/1999509704 >> 172.21.15.12:0/1999509704 conn(0x7f7d200cb9f0 msgr2=0x7f7d200b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:19.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.146+0000 7f7d25745700 1 -- 172.21.15.12:0/1999509704 shutdown_connections 2023-07-08T12:38:19.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.146+0000 7f7d25745700 1 -- 172.21.15.12:0/1999509704 wait complete. 2023-07-08T12:38:19.160 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-08T12:38:05.660194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-08T12:38:06.920177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-08T12:38:10.592749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-08T12:38:19.162 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:38:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.286+0000 7f7b40688700 1 Processor -- start 2023-07-08T12:38:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.286+0000 7f7b40688700 1 -- start start 2023-07-08T12:38:19.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.287+0000 7f7b40688700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7b3806ebe0 0x7f7b380cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.287+0000 7f7b40688700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7b380cd9b0 0x7f7b380cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.287+0000 7f7b40688700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3815d770 con 0x7f7b380aeec0 2023-07-08T12:38:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.287+0000 7f7b40688700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b3806d700 con 0x7f7b380cd9b0 2023-07-08T12:38:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.287+0000 7f7b40688700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3815a600 con 0x7f7b3806ebe0 2023-07-08T12:38:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3ec25700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7b380cd9b0 0x7f7b380cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3e424700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f7b380aeec0 0x7f7b3806cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36666/0 (socket says 172.21.15.12:36666) 2023-07-08T12:38:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3e424700 1 -- 172.21.15.12:0/159443687 learned_addr learned my addr 172.21.15.12:0/159443687 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3dc23700 1 --2- 172.21.15.12:0/159443687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7b3806ebe0 0x7f7b380cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3d422700 1 -- 172.21.15.12:0/159443687 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385389085 0 0) 0x7f7b3815d770 con 0x7f7b380aeec0 2023-07-08T12:38:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3d422700 1 -- 172.21.15.12:0/159443687 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b240035b0 con 0x7f7b380aeec0 2023-07-08T12:38:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3ec25700 1 -- 172.21.15.12:0/159443687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7b3806ebe0 msgr2=0x7f7b380cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3ec25700 1 --2- 172.21.15.12:0/159443687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7b3806ebe0 0x7f7b380cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3ec25700 1 -- 172.21.15.12:0/159443687 >> v1:172.21.15.12:6789/0 conn(0x7f7b380aeec0 legacy=0x7f7b3806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3ec25700 1 -- 172.21.15.12:0/159443687 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b380cddd0 con 0x7f7b380cd9b0 2023-07-08T12:38:19.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3dc23700 1 --2- 172.21.15.12:0/159443687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7b3806ebe0 0x7f7b380cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:38:19.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3ec25700 1 --2- 172.21.15.12:0/159443687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b380cd9b0 0x7f7b380cbe10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009d90 tx=0x7f7b3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f59fc205c094dc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:19.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3d422700 1 -- 172.21.15.12:0/159443687 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3000e070 con 0x7f7b380cd9b0 2023-07-08T12:38:19.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3d422700 1 -- 172.21.15.12:0/159443687 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3000bb20 con 0x7f7b380cd9b0 2023-07-08T12:38:19.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.288+0000 7f7b3d422700 1 -- 172.21.15.12:0/159443687 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3000be90 con 0x7f7b380cd9b0 2023-07-08T12:38:19.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.289+0000 7f7b40688700 1 -- 172.21.15.12:0/159443687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b380cd9b0 msgr2=0x7f7b380cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.289+0000 7f7b40688700 1 --2- 172.21.15.12:0/159443687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b380cd9b0 0x7f7b380cbe10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009d90 tx=0x7f7b3000b330 comp rx=0 tx=0).stop 2023-07-08T12:38:19.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.289+0000 7f7b40688700 1 -- 172.21.15.12:0/159443687 shutdown_connections 2023-07-08T12:38:19.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.289+0000 7f7b40688700 1 --2- 172.21.15.12:0/159443687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b380cd9b0 0x7f7b380cbe10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.289+0000 7f7b40688700 1 --2- 172.21.15.12:0/159443687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7b3806ebe0 0x7f7b380cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.289+0000 7f7b40688700 1 -- 172.21.15.12:0/159443687 >> 172.21.15.12:0/159443687 conn(0x7f7b38098b50 msgr2=0x7f7b3806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:19.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.289+0000 7f7b40688700 1 -- 172.21.15.12:0/159443687 shutdown_connections 2023-07-08T12:38:19.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.289+0000 7f7b40688700 1 -- 172.21.15.12:0/159443687 wait complete. 2023-07-08T12:38:19.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.290+0000 7f7b40688700 1 Processor -- start 2023-07-08T12:38:19.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.290+0000 7f7b40688700 1 -- start start 2023-07-08T12:38:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b40688700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b3806ebe0 0x7f7b380ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b40688700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7b380aeec0 0x7f7b380c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b40688700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b380cd9b0 0x7f7b380b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b40688700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b380b2790 con 0x7f7b380cd9b0 2023-07-08T12:38:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b40688700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b380b2a70 con 0x7f7b3806ebe0 2023-07-08T12:38:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b40688700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b380b2d50 con 0x7f7b380aeec0 2023-07-08T12:38:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b3ec25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b380cd9b0 0x7f7b380b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b3ec25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b380cd9b0 0x7f7b380b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49280/0 (socket says 172.21.15.12:49280) 2023-07-08T12:38:19.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b3ec25700 1 -- 172.21.15.12:0/3671278831 learned_addr learned my addr 172.21.15.12:0/3671278831 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:19.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b3dc23700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7b380aeec0 0x7f7b380c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b3e424700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b3806ebe0 0x7f7b380ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b3ec25700 1 -- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7b380aeec0 msgr2=0x7f7b380c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b3ec25700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7b380aeec0 0x7f7b380c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b3ec25700 1 -- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b3806ebe0 msgr2=0x7f7b380ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b3ec25700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b3806ebe0 0x7f7b380ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b3ec25700 1 -- 172.21.15.12:0/3671278831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b280094a0 con 0x7f7b380cd9b0 2023-07-08T12:38:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b3e424700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b3806ebe0 0x7f7b380ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b3dc23700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7b380aeec0 0x7f7b380c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.291+0000 7f7b3ec25700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b380cd9b0 0x7f7b380b3830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7b30014040 tx=0x7f7b30004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.292+0000 7f7b2effd700 1 -- 172.21.15.12:0/3671278831 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30007000 con 0x7f7b380cd9b0 2023-07-08T12:38:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.292+0000 7f7b2effd700 1 -- 172.21.15.12:0/3671278831 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b30012860 con 0x7f7b380cd9b0 2023-07-08T12:38:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.292+0000 7f7b40688700 1 -- 172.21.15.12:0/3671278831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b30009a70 con 0x7f7b380cd9b0 2023-07-08T12:38:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.292+0000 7f7b2effd700 1 -- 172.21.15.12:0/3671278831 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30012c10 con 0x7f7b380cd9b0 2023-07-08T12:38:19.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.292+0000 7f7b40688700 1 -- 172.21.15.12:0/3671278831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b380b3310 con 0x7f7b380cd9b0 2023-07-08T12:38:19.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.292+0000 7f7b40688700 1 -- 172.21.15.12:0/3671278831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b380c9bd0 con 0x7f7b380cd9b0 2023-07-08T12:38:19.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.293+0000 7f7b2effd700 1 -- 172.21.15.12:0/3671278831 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7b3001c400 con 0x7f7b380cd9b0 2023-07-08T12:38:19.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.293+0000 7f7b2effd700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7b1c06ace0 0x7f7b1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.293+0000 7f7b2effd700 1 -- 172.21.15.12:0/3671278831 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7b3008b680 con 0x7f7b380cd9b0 2023-07-08T12:38:19.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.294+0000 7f7b3e424700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7b1c06ace0 0x7f7b1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.294+0000 7f7b3e424700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7b1c06ace0 0x7f7b1c06d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b34001360 tx=0x7f7b34009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:19.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.298+0000 7f7b2effd700 1 -- 172.21.15.12:0/3671278831 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7b300565f0 con 0x7f7b380cd9b0 2023-07-08T12:38:19.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.499+0000 7f7b40688700 1 -- 172.21.15.12:0/3671278831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b3804ea60 con 0x7f7b380cd9b0 2023-07-08T12:38:19.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.499+0000 7f7b2effd700 1 -- 172.21.15.12:0/3671278831 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b30059c10 con 0x7f7b380cd9b0 2023-07-08T12:38:19.503 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:38:19.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.501+0000 7f7b40688700 1 -- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7b1c06ace0 msgr2=0x7f7b1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.501+0000 7f7b40688700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7b1c06ace0 0x7f7b1c06d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b34001360 tx=0x7f7b34009040 comp rx=0 tx=0).stop 2023-07-08T12:38:19.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.501+0000 7f7b40688700 1 -- 172.21.15.12:0/3671278831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b380cd9b0 msgr2=0x7f7b380b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.501+0000 7f7b40688700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b380cd9b0 0x7f7b380b3830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7b30014040 tx=0x7f7b30004ad0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.501+0000 7f7b40688700 1 -- 172.21.15.12:0/3671278831 shutdown_connections 2023-07-08T12:38:19.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.501+0000 7f7b40688700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7b1c06ace0 0x7f7b1c06d190 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.501+0000 7f7b40688700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b380cd9b0 0x7f7b380b3830 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.501+0000 7f7b40688700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b3806ebe0 0x7f7b380ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.501+0000 7f7b40688700 1 --2- 172.21.15.12:0/3671278831 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7b380aeec0 0x7f7b380c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.501+0000 7f7b40688700 1 -- 172.21.15.12:0/3671278831 >> 172.21.15.12:0/3671278831 conn(0x7f7b38098b50 msgr2=0x7f7b3806cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:19.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.502+0000 7f7b40688700 1 -- 172.21.15.12:0/3671278831 shutdown_connections 2023-07-08T12:38:19.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.502+0000 7f7b40688700 1 -- 172.21.15.12:0/3671278831 wait complete. 2023-07-08T12:38:19.516 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:38:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.642+0000 7fc535d82700 1 Processor -- start 2023-07-08T12:38:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.642+0000 7fc535d82700 1 -- start start 2023-07-08T12:38:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc535d82700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc53006e0c0 0x7fc5301608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc535d82700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc5300cc240 0x7fc5300cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc535d82700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5300ccbe0 con 0x7fc530156a10 2023-07-08T12:38:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc535d82700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc530160410 con 0x7fc53006e0c0 2023-07-08T12:38:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc535d82700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc530156de0 con 0x7fc5300cc240 2023-07-08T12:38:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc52effd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc53006e0c0 0x7fc5301608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc52effd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc53006e0c0 0x7fc5301608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49286/0 (socket says 172.21.15.12:49286) 2023-07-08T12:38:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc52effd700 1 -- 172.21.15.12:0/1705627303 learned_addr learned my addr 172.21.15.12:0/1705627303 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc52ffff700 1 --2- 172.21.15.12:0/1705627303 >> v2:172.21.15.160:3300/0 conn(0x7fc5300cc240 0x7fc5300cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc52effd700 1 -- 172.21.15.12:0/1705627303 >> v2:172.21.15.160:3300/0 conn(0x7fc5300cc240 msgr2=0x7fc5300cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc52effd700 1 --2- 172.21.15.12:0/1705627303 >> v2:172.21.15.160:3300/0 conn(0x7fc5300cc240 0x7fc5300cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc52effd700 1 -- 172.21.15.12:0/1705627303 >> v1:172.21.15.160:6789/0 conn(0x7fc530156a10 legacy=0x7fc53015d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc52effd700 1 -- 172.21.15.12:0/1705627303 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53006ec70 con 0x7fc53006e0c0 2023-07-08T12:38:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc52effd700 1 --2- 172.21.15.12:0/1705627303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc53006e0c0 0x7fc5301608a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc53006c880 tx=0x7fc524009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9d7479221f3e3f server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc52e7fc700 1 -- 172.21.15.12:0/1705627303 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400b870 con 0x7fc53006e0c0 2023-07-08T12:38:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc52e7fc700 1 -- 172.21.15.12:0/1705627303 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52400b9d0 con 0x7fc53006e0c0 2023-07-08T12:38:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.643+0000 7fc52e7fc700 1 -- 172.21.15.12:0/1705627303 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400bd60 con 0x7fc53006e0c0 2023-07-08T12:38:19.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.644+0000 7fc535d82700 1 -- 172.21.15.12:0/1705627303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc53006e0c0 msgr2=0x7fc5301608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.644+0000 7fc535d82700 1 --2- 172.21.15.12:0/1705627303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc53006e0c0 0x7fc5301608a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc53006c880 tx=0x7fc524009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.644+0000 7fc535d82700 1 -- 172.21.15.12:0/1705627303 shutdown_connections 2023-07-08T12:38:19.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.644+0000 7fc535d82700 1 --2- 172.21.15.12:0/1705627303 >> v2:172.21.15.160:3300/0 conn(0x7fc5300cc240 0x7fc5300cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.644+0000 7fc535d82700 1 --2- 172.21.15.12:0/1705627303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc53006e0c0 0x7fc5301608a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.644+0000 7fc535d82700 1 -- 172.21.15.12:0/1705627303 >> 172.21.15.12:0/1705627303 conn(0x7fc530098b50 msgr2=0x7fc53006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:19.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.644+0000 7fc535d82700 1 -- 172.21.15.12:0/1705627303 shutdown_connections 2023-07-08T12:38:19.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.644+0000 7fc535d82700 1 -- 172.21.15.12:0/1705627303 wait complete. 2023-07-08T12:38:19.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.645+0000 7fc535d82700 1 Processor -- start 2023-07-08T12:38:19.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.645+0000 7fc535d82700 1 -- start start 2023-07-08T12:38:19.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.646+0000 7fc535d82700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc53006e0c0 0x7fc5300b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.646+0000 7fc535d82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5300cc240 0x7fc5300994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.646+0000 7fc535d82700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc530156a10 0x7fc530099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.646+0000 7fc535d82700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53009b3d0 con 0x7fc5300cc240 2023-07-08T12:38:19.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.646+0000 7fc535d82700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53009b670 con 0x7fc530156a10 2023-07-08T12:38:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.646+0000 7fc535d82700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53009b950 con 0x7fc53006e0c0 2023-07-08T12:38:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.646+0000 7fc52effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5300cc240 0x7fc5300994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.646+0000 7fc52effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5300cc240 0x7fc5300994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49292/0 (socket says 172.21.15.12:49292) 2023-07-08T12:38:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.646+0000 7fc52effd700 1 -- 172.21.15.12:0/10933658 learned_addr learned my addr 172.21.15.12:0/10933658 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.646+0000 7fc52ffff700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc530156a10 0x7fc530099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.646+0000 7fc52f7fe700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc53006e0c0 0x7fc5300b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.647+0000 7fc52effd700 1 -- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc53006e0c0 msgr2=0x7fc5300b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.647+0000 7fc52effd700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc53006e0c0 0x7fc5300b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.647+0000 7fc52effd700 1 -- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc530156a10 msgr2=0x7fc530099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.647+0000 7fc52effd700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc530156a10 0x7fc530099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.647+0000 7fc52effd700 1 -- 172.21.15.12:0/10933658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc524009550 con 0x7fc5300cc240 2023-07-08T12:38:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.647+0000 7fc52effd700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5300cc240 0x7fc5300994c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc524000c00 tx=0x7fc524004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.647+0000 7fc517fff700 1 -- 172.21.15.12:0/10933658 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524004540 con 0x7fc5300cc240 2023-07-08T12:38:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.647+0000 7fc517fff700 1 -- 172.21.15.12:0/10933658 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5240036a0 con 0x7fc5300cc240 2023-07-08T12:38:19.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.647+0000 7fc535d82700 1 -- 172.21.15.12:0/10933658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53009bc30 con 0x7fc5300cc240 2023-07-08T12:38:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.647+0000 7fc517fff700 1 -- 172.21.15.12:0/10933658 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524003a30 con 0x7fc5300cc240 2023-07-08T12:38:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.647+0000 7fc535d82700 1 -- 172.21.15.12:0/10933658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53009c1f0 con 0x7fc5300cc240 2023-07-08T12:38:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.648+0000 7fc515ffb700 1 -- 172.21.15.12:0/10933658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5080052f0 con 0x7fc5300cc240 2023-07-08T12:38:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.648+0000 7fc517fff700 1 -- 172.21.15.12:0/10933658 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc524021070 con 0x7fc5300cc240 2023-07-08T12:38:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.649+0000 7fc517fff700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc51c06ace0 0x7fc51c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.649+0000 7fc517fff700 1 -- 172.21.15.12:0/10933658 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc52408aed0 con 0x7fc5300cc240 2023-07-08T12:38:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.649+0000 7fc52f7fe700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc51c06ace0 0x7fc51c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.650+0000 7fc52f7fe700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc51c06ace0 0x7fc51c06d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc520000fb0 tx=0x7fc520009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:19.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.653+0000 7fc517fff700 1 -- 172.21.15.12:0/10933658 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc524055d20 con 0x7fc5300cc240 2023-07-08T12:38:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.849+0000 7fc515ffb700 1 -- 172.21.15.12:0/10933658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc508005b20 con 0x7fc5300cc240 2023-07-08T12:38:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.850+0000 7fc517fff700 1 -- 172.21.15.12:0/10933658 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc5240150a0 con 0x7fc5300cc240 2023-07-08T12:38:19.853 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:38:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.852+0000 7fc515ffb700 1 -- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc51c06ace0 msgr2=0x7fc51c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.852+0000 7fc515ffb700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc51c06ace0 0x7fc51c06d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc520000fb0 tx=0x7fc520009040 comp rx=0 tx=0).stop 2023-07-08T12:38:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.852+0000 7fc515ffb700 1 -- 172.21.15.12:0/10933658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5300cc240 msgr2=0x7fc5300994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.852+0000 7fc515ffb700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5300cc240 0x7fc5300994c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc524000c00 tx=0x7fc524004750 comp rx=0 tx=0).stop 2023-07-08T12:38:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.852+0000 7fc515ffb700 1 -- 172.21.15.12:0/10933658 shutdown_connections 2023-07-08T12:38:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.852+0000 7fc515ffb700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc51c06ace0 0x7fc51c06d190 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.852+0000 7fc515ffb700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc530156a10 0x7fc530099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.852+0000 7fc515ffb700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc53006e0c0 0x7fc5300b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.852+0000 7fc515ffb700 1 --2- 172.21.15.12:0/10933658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5300cc240 0x7fc5300994c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.852+0000 7fc515ffb700 1 -- 172.21.15.12:0/10933658 >> 172.21.15.12:0/10933658 conn(0x7fc530098b50 msgr2=0x7fc53015f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.852+0000 7fc515ffb700 1 -- 172.21.15.12:0/10933658 shutdown_connections 2023-07-08T12:38:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.852+0000 7fc515ffb700 1 -- 172.21.15.12:0/10933658 wait complete. 2023-07-08T12:38:19.866 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-08T12:38:19.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.992+0000 7f90cb11e700 1 Processor -- start 2023-07-08T12:38:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.992+0000 7f90cb11e700 1 -- start start 2023-07-08T12:38:19.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.993+0000 7f90cb11e700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f90c4158a00 0x7f90c415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.993+0000 7f90cb11e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90c40cd5d0 0x7f90c406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.993+0000 7f90cb11e700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90c406b510 0x7f90c4070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:19.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.993+0000 7f90cb11e700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c40cc910 con 0x7f90c406b510 2023-07-08T12:38:19.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.993+0000 7f90cb11e700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c4069f30 con 0x7f90c4158a00 2023-07-08T12:38:19.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.993+0000 7f90cb11e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c406a210 con 0x7f90c40cd5d0 2023-07-08T12:38:19.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c96bb700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90c406b510 0x7f90c4070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c96bb700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90c406b510 0x7f90c4070ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49302/0 (socket says 172.21.15.12:49302) 2023-07-08T12:38:19.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c96bb700 1 -- 172.21.15.12:0/3002787571 learned_addr learned my addr 172.21.15.12:0/3002787571 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:19.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c8eba700 1 --2- 172.21.15.12:0/3002787571 >> v2:172.21.15.160:3300/0 conn(0x7f90c4158a00 0x7f90c415cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c3fff700 1 --2- 172.21.15.12:0/3002787571 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90c40cd5d0 0x7f90c406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c96bb700 1 -- 172.21.15.12:0/3002787571 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90c40cd5d0 msgr2=0x7f90c406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c96bb700 1 --2- 172.21.15.12:0/3002787571 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90c40cd5d0 0x7f90c406afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c96bb700 1 -- 172.21.15.12:0/3002787571 >> v2:172.21.15.160:3300/0 conn(0x7f90c4158a00 msgr2=0x7f90c415cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c96bb700 1 --2- 172.21.15.12:0/3002787571 >> v2:172.21.15.160:3300/0 conn(0x7f90c4158a00 0x7f90c415cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c96bb700 1 -- 172.21.15.12:0/3002787571 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c406a4f0 con 0x7f90c406b510 2023-07-08T12:38:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c96bb700 1 --2- 172.21.15.12:0/3002787571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90c406b510 0x7f90c4070ae0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f90c40719a0 tx=0x7f90bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fe91ce5835e706 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c37fe700 1 -- 172.21.15.12:0/3002787571 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc00e070 con 0x7f90c406b510 2023-07-08T12:38:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c37fe700 1 -- 172.21.15.12:0/3002787571 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc00b910 con 0x7f90c406b510 2023-07-08T12:38:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.994+0000 7f90c37fe700 1 -- 172.21.15.12:0/3002787571 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc00bcc0 con 0x7f90c406b510 2023-07-08T12:38:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.995+0000 7f90cb11e700 1 -- 172.21.15.12:0/3002787571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90c406b510 msgr2=0x7f90c4070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.995+0000 7f90cb11e700 1 --2- 172.21.15.12:0/3002787571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90c406b510 0x7f90c4070ae0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f90c40719a0 tx=0x7f90bc00b330 comp rx=0 tx=0).stop 2023-07-08T12:38:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.995+0000 7f90cb11e700 1 -- 172.21.15.12:0/3002787571 shutdown_connections 2023-07-08T12:38:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.995+0000 7f90cb11e700 1 --2- 172.21.15.12:0/3002787571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90c406b510 0x7f90c4070ae0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.995+0000 7f90cb11e700 1 --2- 172.21.15.12:0/3002787571 >> v2:172.21.15.160:3300/0 conn(0x7f90c4158a00 0x7f90c415cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.995+0000 7f90cb11e700 1 --2- 172.21.15.12:0/3002787571 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90c40cd5d0 0x7f90c406afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.995+0000 7f90cb11e700 1 -- 172.21.15.12:0/3002787571 >> 172.21.15.12:0/3002787571 conn(0x7f90c40cf1d0 msgr2=0x7f90c40cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.995+0000 7f90cb11e700 1 -- 172.21.15.12:0/3002787571 shutdown_connections 2023-07-08T12:38:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.995+0000 7f90cb11e700 1 -- 172.21.15.12:0/3002787571 wait complete. 2023-07-08T12:38:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.996+0000 7f90cb11e700 1 Processor -- start 2023-07-08T12:38:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.997+0000 7f90cb11e700 1 -- start start 2023-07-08T12:38:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90cb11e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90c406b510 0x7f90c40b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90cb11e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90c40cd5d0 0x7f90c41f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90cb11e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90c4158a00 0x7f90c414c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90cb11e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c41f3920 con 0x7f90c4158a00 2023-07-08T12:38:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90cb11e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c4158dd0 con 0x7f90c40cd5d0 2023-07-08T12:38:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90cb11e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40b16f0 con 0x7f90c406b510 2023-07-08T12:38:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90c96bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90c4158a00 0x7f90c414c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90c96bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90c4158a00 0x7f90c414c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49308/0 (socket says 172.21.15.12:49308) 2023-07-08T12:38:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90c96bb700 1 -- 172.21.15.12:0/1722024729 learned_addr learned my addr 172.21.15.12:0/1722024729 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90c8eba700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90c406b510 0x7f90c40b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90c3fff700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90c40cd5d0 0x7f90c41f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90c96bb700 1 -- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90c406b510 msgr2=0x7f90c40b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90c96bb700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90c406b510 0x7f90c40b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90c96bb700 1 -- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90c40cd5d0 msgr2=0x7f90c41f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90c96bb700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90c40cd5d0 0x7f90c41f33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90c96bb700 1 -- 172.21.15.12:0/1722024729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc009b30 con 0x7f90c4158a00 2023-07-08T12:38:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90c96bb700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90c4158a00 0x7f90c414c0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f90c406f1a0 tx=0x7f90bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90c17fa700 1 -- 172.21.15.12:0/1722024729 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc0099b0 con 0x7f90c4158a00 2023-07-08T12:38:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.998+0000 7f90c17fa700 1 -- 172.21.15.12:0/1722024729 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc00ce80 con 0x7f90c4158a00 2023-07-08T12:38:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.999+0000 7f90cb11e700 1 -- 172.21.15.12:0/1722024729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c41f3b50 con 0x7f90c4158a00 2023-07-08T12:38:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.999+0000 7f90c17fa700 1 -- 172.21.15.12:0/1722024729 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc004490 con 0x7f90c4158a00 2023-07-08T12:38:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:19.999+0000 7f90cb11e700 1 -- 172.21.15.12:0/1722024729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c41f4110 con 0x7f90c4158a00 2023-07-08T12:38:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.000+0000 7f90cb11e700 1 -- 172.21.15.12:0/1722024729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c40cd440 con 0x7f90c4158a00 2023-07-08T12:38:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.000+0000 7f90c17fa700 1 -- 172.21.15.12:0/1722024729 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f90bc0045f0 con 0x7f90c4158a00 2023-07-08T12:38:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.000+0000 7f90c17fa700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f90a806ace0 0x7f90a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.000+0000 7f90c17fa700 1 -- 172.21.15.12:0/1722024729 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f90bc08a9e0 con 0x7f90c4158a00 2023-07-08T12:38:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.000+0000 7f90c8eba700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f90a806ace0 0x7f90a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.001+0000 7f90c8eba700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f90a806ace0 0x7f90a806d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009b40 tx=0x7f90b8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.005+0000 7f90c17fa700 1 -- 172.21.15.12:0/1722024729 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f90bc055830 con 0x7f90c4158a00 2023-07-08T12:38:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.202+0000 7f90cb11e700 1 -- 172.21.15.12:0/1722024729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90c404ea60 con 0x7f90c4158a00 2023-07-08T12:38:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.202+0000 7f90c17fa700 1 -- 172.21.15.12:0/1722024729 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f90bc058e50 con 0x7f90c4158a00 2023-07-08T12:38:20.205 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:38:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.204+0000 7f90cb11e700 1 -- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f90a806ace0 msgr2=0x7f90a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.204+0000 7f90cb11e700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f90a806ace0 0x7f90a806d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009b40 tx=0x7f90b8009210 comp rx=0 tx=0).stop 2023-07-08T12:38:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.204+0000 7f90cb11e700 1 -- 172.21.15.12:0/1722024729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90c4158a00 msgr2=0x7f90c414c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.204+0000 7f90cb11e700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90c4158a00 0x7f90c414c0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f90c406f1a0 tx=0x7f90bc004ad0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.204+0000 7f90cb11e700 1 -- 172.21.15.12:0/1722024729 shutdown_connections 2023-07-08T12:38:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.204+0000 7f90cb11e700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f90a806ace0 0x7f90a806d190 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.204+0000 7f90cb11e700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90c4158a00 0x7f90c414c0a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.204+0000 7f90cb11e700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90c406b510 0x7f90c40b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.204+0000 7f90cb11e700 1 --2- 172.21.15.12:0/1722024729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90c40cd5d0 0x7f90c41f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.204+0000 7f90cb11e700 1 -- 172.21.15.12:0/1722024729 >> 172.21.15.12:0/1722024729 conn(0x7f90c40cf1d0 msgr2=0x7f90c4065cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.204+0000 7f90cb11e700 1 -- 172.21.15.12:0/1722024729 shutdown_connections 2023-07-08T12:38:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.204+0000 7f90cb11e700 1 -- 172.21.15.12:0/1722024729 wait complete. 2023-07-08T12:38:20.219 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-08T12:38:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.345+0000 7f3e8815c700 1 Processor -- start 2023-07-08T12:38:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.345+0000 7f3e8815c700 1 -- start start 2023-07-08T12:38:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e8815c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3e800cb640 0x7f3e801607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e8815c700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3e800b1920 0x7f3e80069fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e8815c700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e800ceda0 con 0x7f3e800cd360 2023-07-08T12:38:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e8815c700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e80160d10 con 0x7f3e800cb640 2023-07-08T12:38:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e8815c700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e800cefd0 con 0x7f3e800b1920 2023-07-08T12:38:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e85ef8700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3e800cb640 0x7f3e801607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e856f7700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3e800cd360 0x7f3e800b1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36708/0 (socket says 172.21.15.12:36708) 2023-07-08T12:38:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e856f7700 1 -- 172.21.15.12:0/3939762944 learned_addr learned my addr 172.21.15.12:0/3939762944 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e866f9700 1 --2- 172.21.15.12:0/3939762944 >> v2:172.21.15.160:3300/0 conn(0x7f3e800b1920 0x7f3e80069fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e84ef6700 1 -- 172.21.15.12:0/3939762944 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492415038 0 0) 0x7f3e800ceda0 con 0x7f3e800cd360 2023-07-08T12:38:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e84ef6700 1 -- 172.21.15.12:0/3939762944 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e6c0035b0 con 0x7f3e800cd360 2023-07-08T12:38:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e85ef8700 1 -- 172.21.15.12:0/3939762944 >> v2:172.21.15.160:3300/0 conn(0x7f3e800b1920 msgr2=0x7f3e80069fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e85ef8700 1 --2- 172.21.15.12:0/3939762944 >> v2:172.21.15.160:3300/0 conn(0x7f3e800b1920 0x7f3e80069fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e85ef8700 1 -- 172.21.15.12:0/3939762944 >> v1:172.21.15.12:6789/0 conn(0x7f3e800cd360 legacy=0x7f3e800b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e85ef8700 1 -- 172.21.15.12:0/3939762944 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e800cf590 con 0x7f3e800cb640 2023-07-08T12:38:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e866f9700 1 --2- 172.21.15.12:0/3939762944 >> v2:172.21.15.160:3300/0 conn(0x7f3e800b1920 0x7f3e80069fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e85ef8700 1 --2- 172.21.15.12:0/3939762944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cb640 0x7f3e801607d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3e8006ea00 tx=0x7f3e7c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f950e0a8745afc14 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.346+0000 7f3e84ef6700 1 -- 172.21.15.12:0/3939762944 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c011830 con 0x7f3e800cb640 2023-07-08T12:38:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.347+0000 7f3e84ef6700 1 -- 172.21.15.12:0/3939762944 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7c011990 con 0x7f3e800cb640 2023-07-08T12:38:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.347+0000 7f3e84ef6700 1 -- 172.21.15.12:0/3939762944 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c011d20 con 0x7f3e800cb640 2023-07-08T12:38:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.347+0000 7f3e8815c700 1 -- 172.21.15.12:0/3939762944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cb640 msgr2=0x7f3e801607d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.347+0000 7f3e8815c700 1 --2- 172.21.15.12:0/3939762944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cb640 0x7f3e801607d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3e8006ea00 tx=0x7f3e7c009b20 comp rx=0 tx=0).stop 2023-07-08T12:38:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.347+0000 7f3e8815c700 1 -- 172.21.15.12:0/3939762944 shutdown_connections 2023-07-08T12:38:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.347+0000 7f3e8815c700 1 --2- 172.21.15.12:0/3939762944 >> v2:172.21.15.160:3300/0 conn(0x7f3e800b1920 0x7f3e80069fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.347+0000 7f3e8815c700 1 --2- 172.21.15.12:0/3939762944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cb640 0x7f3e801607d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.347+0000 7f3e8815c700 1 -- 172.21.15.12:0/3939762944 >> 172.21.15.12:0/3939762944 conn(0x7f3e800cc210 msgr2=0x7f3e800cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.347+0000 7f3e8815c700 1 -- 172.21.15.12:0/3939762944 shutdown_connections 2023-07-08T12:38:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.347+0000 7f3e8815c700 1 -- 172.21.15.12:0/3939762944 wait complete. 2023-07-08T12:38:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.348+0000 7f3e8815c700 1 Processor -- start 2023-07-08T12:38:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.348+0000 7f3e8815c700 1 -- start start 2023-07-08T12:38:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.349+0000 7f3e8815c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e800b1920 0x7f3e80149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.349+0000 7f3e8815c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cb640 0x7f3e8015d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.349+0000 7f3e8815c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e800cd360 0x7f3e8015e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.349+0000 7f3e8815c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8015cf30 con 0x7f3e800cb640 2023-07-08T12:38:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.349+0000 7f3e8815c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8015c8e0 con 0x7f3e800b1920 2023-07-08T12:38:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.349+0000 7f3e8815c700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8015ec40 con 0x7f3e800cd360 2023-07-08T12:38:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.349+0000 7f3e856f7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cb640 0x7f3e8015d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.349+0000 7f3e856f7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cb640 0x7f3e8015d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49320/0 (socket says 172.21.15.12:49320) 2023-07-08T12:38:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.349+0000 7f3e856f7700 1 -- 172.21.15.12:0/776345643 learned_addr learned my addr 172.21.15.12:0/776345643 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.349+0000 7f3e85ef8700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e800b1920 0x7f3e80149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.349+0000 7f3e866f9700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e800cd360 0x7f3e8015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.350+0000 7f3e856f7700 1 -- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e800cd360 msgr2=0x7f3e8015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.350+0000 7f3e856f7700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e800cd360 0x7f3e8015e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.350+0000 7f3e856f7700 1 -- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e800b1920 msgr2=0x7f3e80149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.350+0000 7f3e856f7700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e800b1920 0x7f3e80149cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.350+0000 7f3e856f7700 1 -- 172.21.15.12:0/776345643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c0094a0 con 0x7f3e800cb640 2023-07-08T12:38:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.350+0000 7f3e856f7700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cb640 0x7f3e8015d350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000c4d0 tx=0x7f3e7000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.350+0000 7f3e767fc700 1 -- 172.21.15.12:0/776345643 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70005d70 con 0x7f3e800cb640 2023-07-08T12:38:20.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.350+0000 7f3e767fc700 1 -- 172.21.15.12:0/776345643 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e700070d0 con 0x7f3e800cb640 2023-07-08T12:38:20.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.350+0000 7f3e8815c700 1 -- 172.21.15.12:0/776345643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8015cb50 con 0x7f3e800cb640 2023-07-08T12:38:20.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.350+0000 7f3e767fc700 1 -- 172.21.15.12:0/776345643 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70007460 con 0x7f3e800cb640 2023-07-08T12:38:20.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.350+0000 7f3e8815c700 1 -- 172.21.15.12:0/776345643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8015d890 con 0x7f3e800cb640 2023-07-08T12:38:20.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.351+0000 7f3e8815c700 1 -- 172.21.15.12:0/776345643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8004ea60 con 0x7f3e800cb640 2023-07-08T12:38:20.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.351+0000 7f3e767fc700 1 -- 172.21.15.12:0/776345643 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e700075c0 con 0x7f3e800cb640 2023-07-08T12:38:20.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.352+0000 7f3e767fc700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e6406ace0 0x7f3e6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.352+0000 7f3e767fc700 1 -- 172.21.15.12:0/776345643 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3e70088c40 con 0x7f3e800cb640 2023-07-08T12:38:20.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.352+0000 7f3e85ef8700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e6406ace0 0x7f3e6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.352+0000 7f3e85ef8700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e6406ace0 0x7f3e6406d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0090b0 tx=0x7f3e7c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:20.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.356+0000 7f3e767fc700 1 -- 172.21.15.12:0/776345643 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3e70053a00 con 0x7f3e800cb640 2023-07-08T12:38:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.552+0000 7f3e8815c700 1 -- 172.21.15.12:0/776345643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e80071c80 con 0x7f3e800cb640 2023-07-08T12:38:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.552+0000 7f3e767fc700 1 -- 172.21.15.12:0/776345643 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3e70057020 con 0x7f3e800cb640 2023-07-08T12:38:20.555 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-07-08T12:38:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.554+0000 7f3e8815c700 1 -- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e6406ace0 msgr2=0x7f3e6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.554+0000 7f3e8815c700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e6406ace0 0x7f3e6406d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0090b0 tx=0x7f3e7c008030 comp rx=0 tx=0).stop 2023-07-08T12:38:20.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.554+0000 7f3e8815c700 1 -- 172.21.15.12:0/776345643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cb640 msgr2=0x7f3e8015d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.554+0000 7f3e8815c700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cb640 0x7f3e8015d350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000c4d0 tx=0x7f3e7000c7e0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.554+0000 7f3e8815c700 1 -- 172.21.15.12:0/776345643 shutdown_connections 2023-07-08T12:38:20.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.554+0000 7f3e8815c700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e6406ace0 0x7f3e6406d190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.554+0000 7f3e8815c700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e800cd360 0x7f3e8015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.554+0000 7f3e8815c700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e800b1920 0x7f3e80149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.554+0000 7f3e8815c700 1 --2- 172.21.15.12:0/776345643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cb640 0x7f3e8015d350 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.554+0000 7f3e8815c700 1 -- 172.21.15.12:0/776345643 >> 172.21.15.12:0/776345643 conn(0x7f3e800cc210 msgr2=0x7f3e800c5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:20.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.554+0000 7f3e8815c700 1 -- 172.21.15.12:0/776345643 shutdown_connections 2023-07-08T12:38:20.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.554+0000 7f3e8815c700 1 -- 172.21.15.12:0/776345643 wait complete. 2023-07-08T12:38:20.568 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-08T12:38:20.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.695+0000 7f831b73d700 1 Processor -- start 2023-07-08T12:38:20.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.695+0000 7f831b73d700 1 -- start start 2023-07-08T12:38:20.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.696+0000 7f831b73d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f831406e0c0 0x7f83141608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.696+0000 7f831b73d700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8314160410 con 0x7f83140cc240 2023-07-08T12:38:20.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.696+0000 7f831b73d700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8314156de0 con 0x7f8314156a10 2023-07-08T12:38:20.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.696+0000 7f831b73d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831406ec70 con 0x7f831406e0c0 2023-07-08T12:38:20.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.696+0000 7f8319cda700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f83140cc240 0x7f83140ae440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36728/0 (socket says 172.21.15.12:36728) 2023-07-08T12:38:20.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.696+0000 7f8319cda700 1 -- 172.21.15.12:0/2247308046 learned_addr learned my addr 172.21.15.12:0/2247308046 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:20.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f8318cd8700 1 --2- 172.21.15.12:0/2247308046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f831406e0c0 0x7f83141608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474505756 0 0) 0x7f8314160410 con 0x7f83140cc240 2023-07-08T12:38:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83000035b0 con 0x7f83140cc240 2023-07-08T12:38:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168001321 0 0) 0x7f8314156de0 con 0x7f8314156a10 2023-07-08T12:38:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8314160410 con 0x7f8314156a10 2023-07-08T12:38:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2365164593 0 0) 0x7f83000035b0 con 0x7f83140cc240 2023-07-08T12:38:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f831406e0c0 msgr2=0x7f83141608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 --2- 172.21.15.12:0/2247308046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f831406e0c0 0x7f83141608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 >> v1:172.21.15.160:6789/0 conn(0x7f8314156a10 legacy=0x7f831415d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831406e6b0 con 0x7f83140cc240 2023-07-08T12:38:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f830c0019f0 con 0x7f83140cc240 2023-07-08T12:38:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f8318cd8700 1 --2- 172.21.15.12:0/2247308046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f831406e0c0 0x7f83141608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:38:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 >> v1:172.21.15.12:6789/0 conn(0x7f83140cc240 legacy=0x7f83140ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 --2- 172.21.15.12:0/2247308046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8300003bc0 0x7f8300003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 --2- 172.21.15.12:0/2247308046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8300004850 0x7f8300004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 --2- 172.21.15.12:0/2247308046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f831406e0c0 0x7f8300005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831406ec70 con 0x7f8300003bc0 2023-07-08T12:38:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830000ba40 con 0x7f8300004850 2023-07-08T12:38:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830000bc00 con 0x7f831406e0c0 2023-07-08T12:38:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f83194d9700 1 --2- 172.21.15.12:0/2247308046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8300003bc0 0x7f8300003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f8318cd8700 1 --2- 172.21.15.12:0/2247308046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8300004850 0x7f8300004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f8319cda700 1 --2- 172.21.15.12:0/2247308046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f831406e0c0 0x7f8300005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f83194d9700 1 -- 172.21.15.12:0/2247308046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f831406e0c0 msgr2=0x7f8300005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f83194d9700 1 --2- 172.21.15.12:0/2247308046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f831406e0c0 0x7f8300005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f83194d9700 1 -- 172.21.15.12:0/2247308046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8300004850 msgr2=0x7f8300004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.697+0000 7f83194d9700 1 --2- 172.21.15.12:0/2247308046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8300004850 0x7f8300004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.698+0000 7f83194d9700 1 -- 172.21.15.12:0/2247308046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83100094a0 con 0x7f8300003bc0 2023-07-08T12:38:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.698+0000 7f83194d9700 1 --2- 172.21.15.12:0/2247308046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8300003bc0 0x7f8300003fb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f83040019e0 tx=0x7f8300005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9e072503585b7a server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.698+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83040014a0 con 0x7f8300003bc0 2023-07-08T12:38:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.698+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83040052d0 con 0x7f8300003bc0 2023-07-08T12:38:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.698+0000 7f830bfff700 1 -- 172.21.15.12:0/2247308046 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304001d50 con 0x7f8300003bc0 2023-07-08T12:38:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.698+0000 7f831b73d700 1 -- 172.21.15.12:0/2247308046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8300003bc0 msgr2=0x7f8300003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.698+0000 7f831b73d700 1 --2- 172.21.15.12:0/2247308046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8300003bc0 0x7f8300003fb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f83040019e0 tx=0x7f8300005880 comp rx=0 tx=0).stop 2023-07-08T12:38:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.698+0000 7f8319cda700 1 -- 172.21.15.12:0/2247308046 reap_dead start 2023-07-08T12:38:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.698+0000 7f831b73d700 1 -- 172.21.15.12:0/2247308046 shutdown_connections 2023-07-08T12:38:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.698+0000 7f831b73d700 1 -- 172.21.15.12:0/2247308046 >> 172.21.15.12:0/2247308046 conn(0x7f8314098b50 msgr2=0x7f831406b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.698+0000 7f831b73d700 1 -- 172.21.15.12:0/2247308046 shutdown_connections 2023-07-08T12:38:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.699+0000 7f831b73d700 1 -- 172.21.15.12:0/2247308046 wait complete. 2023-07-08T12:38:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.700+0000 7f831b73d700 1 Processor -- start 2023-07-08T12:38:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.700+0000 7f831b73d700 1 -- start start 2023-07-08T12:38:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.700+0000 7f831b73d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8300003bc0 0x7f8314159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.700+0000 7f831b73d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83140ce330 0x7f83140ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.700+0000 7f831b73d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f831414b4a0 0x7f831414b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.700+0000 7f831b73d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8314145d10 con 0x7f8300003bc0 2023-07-08T12:38:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.700+0000 7f831b73d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83140cece0 con 0x7f831414b4a0 2023-07-08T12:38:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.700+0000 7f831b73d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8314159880 con 0x7f83140ce330 2023-07-08T12:38:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f83194d9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8300003bc0 0x7f8314159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f83194d9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8300003bc0 0x7f8314159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49340/0 (socket says 172.21.15.12:49340) 2023-07-08T12:38:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f83194d9700 1 -- 172.21.15.12:0/1316981566 learned_addr learned my addr 172.21.15.12:0/1316981566 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f8318cd8700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83140ce330 0x7f83140ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f8319cda700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f831414b4a0 0x7f831414b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f83194d9700 1 -- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83140ce330 msgr2=0x7f83140ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f83194d9700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83140ce330 0x7f83140ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f83194d9700 1 -- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f831414b4a0 msgr2=0x7f831414b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f83194d9700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f831414b4a0 0x7f831414b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f83194d9700 1 -- 172.21.15.12:0/1316981566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831414be60 con 0x7f8300003bc0 2023-07-08T12:38:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f8318cd8700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83140ce330 0x7f83140ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f8319cda700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f831414b4a0 0x7f831414b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f83194d9700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8300003bc0 0x7f8314159d10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f830400c3c0 tx=0x7f8304001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f8309ffb700 1 -- 172.21.15.12:0/1316981566 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304006e70 con 0x7f8300003bc0 2023-07-08T12:38:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f8309ffb700 1 -- 172.21.15.12:0/1316981566 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8304001440 con 0x7f8300003bc0 2023-07-08T12:38:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f8309ffb700 1 -- 172.21.15.12:0/1316981566 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304001d50 con 0x7f8300003bc0 2023-07-08T12:38:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f831b73d700 1 -- 172.21.15.12:0/1316981566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831415f220 con 0x7f8300003bc0 2023-07-08T12:38:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.701+0000 7f831b73d700 1 -- 172.21.15.12:0/1316981566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8314159510 con 0x7f8300003bc0 2023-07-08T12:38:20.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.702+0000 7f831b73d700 1 -- 172.21.15.12:0/1316981566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8314066af0 con 0x7f8300003bc0 2023-07-08T12:38:20.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.703+0000 7f8309ffb700 1 -- 172.21.15.12:0/1316981566 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8304007250 con 0x7f8300003bc0 2023-07-08T12:38:20.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.703+0000 7f8309ffb700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f82f806ace0 0x7f82f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:20.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.703+0000 7f8309ffb700 1 -- 172.21.15.12:0/1316981566 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f830408a510 con 0x7f8300003bc0 2023-07-08T12:38:20.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.704+0000 7f8318cd8700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f82f806ace0 0x7f82f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:20.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.704+0000 7f8318cd8700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f82f806ace0 0x7f82f806d190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8310009ce0 tx=0x7f8310008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:20.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.708+0000 7f8309ffb700 1 -- 172.21.15.12:0/1316981566 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8304058df0 con 0x7f8300003bc0 2023-07-08T12:38:20.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.905+0000 7f831b73d700 1 -- 172.21.15.12:0/1316981566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f831404ea60 con 0x7f8300003bc0 2023-07-08T12:38:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.906+0000 7f8309ffb700 1 -- 172.21.15.12:0/1316981566 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8304021020 con 0x7f8300003bc0 2023-07-08T12:38:20.908 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:38:20.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.907+0000 7f831b73d700 1 -- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f82f806ace0 msgr2=0x7f82f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.908+0000 7f831b73d700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f82f806ace0 0x7f82f806d190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8310009ce0 tx=0x7f8310008040 comp rx=0 tx=0).stop 2023-07-08T12:38:20.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.908+0000 7f831b73d700 1 -- 172.21.15.12:0/1316981566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8300003bc0 msgr2=0x7f8314159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:20.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.908+0000 7f831b73d700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8300003bc0 0x7f8314159d10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f830400c3c0 tx=0x7f8304001360 comp rx=0 tx=0).stop 2023-07-08T12:38:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.908+0000 7f831b73d700 1 -- 172.21.15.12:0/1316981566 shutdown_connections 2023-07-08T12:38:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.908+0000 7f831b73d700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f82f806ace0 0x7f82f806d190 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.908+0000 7f831b73d700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f831414b4a0 0x7f831414b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.908+0000 7f831b73d700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8300003bc0 0x7f8314159d10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.908+0000 7f831b73d700 1 --2- 172.21.15.12:0/1316981566 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83140ce330 0x7f83140ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.908+0000 7f831b73d700 1 -- 172.21.15.12:0/1316981566 >> 172.21.15.12:0/1316981566 conn(0x7f8314098b50 msgr2=0x7f831415b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.908+0000 7f831b73d700 1 -- 172.21.15.12:0/1316981566 shutdown_connections 2023-07-08T12:38:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:20.908+0000 7f831b73d700 1 -- 172.21.15.12:0/1316981566 wait complete. 2023-07-08T12:38:20.922 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-08T12:38:21.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.045+0000 7f3100827700 1 Processor -- start 2023-07-08T12:38:21.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.045+0000 7f3100827700 1 -- start start 2023-07-08T12:38:21.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f3100827700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f30fc0ce1b0 0x7f30fc0cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f3100827700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30fc0b1c70 0x7f30fc0ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f3100827700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f30fc15d6c0 0x7f30fc071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f3100827700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30fc0c9e60 con 0x7f30fc15d6c0 2023-07-08T12:38:21.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f3100827700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30fc15d230 con 0x7f30fc0ce1b0 2023-07-08T12:38:21.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f3100827700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0cd920 con 0x7f30fc0b1c70 2023-07-08T12:38:21.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f30fad9d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f30fc15d6c0 0x7f30fc071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f30fad9d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f30fc15d6c0 0x7f30fc071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49352/0 (socket says 172.21.15.12:49352) 2023-07-08T12:38:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f30fad9d700 1 -- 172.21.15.12:0/1171235815 learned_addr learned my addr 172.21.15.12:0/1171235815 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f30fa59c700 1 --2- 172.21.15.12:0/1171235815 >> v2:172.21.15.160:3300/0 conn(0x7f30fc0ce1b0 0x7f30fc0cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f30f9d9b700 1 --2- 172.21.15.12:0/1171235815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30fc0b1c70 0x7f30fc0ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f30fad9d700 1 -- 172.21.15.12:0/1171235815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30fc0b1c70 msgr2=0x7f30fc0ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f30fad9d700 1 --2- 172.21.15.12:0/1171235815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30fc0b1c70 0x7f30fc0ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f30fad9d700 1 -- 172.21.15.12:0/1171235815 >> v2:172.21.15.160:3300/0 conn(0x7f30fc0ce1b0 msgr2=0x7f30fc0cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f30fad9d700 1 --2- 172.21.15.12:0/1171235815 >> v2:172.21.15.160:3300/0 conn(0x7f30fc0ce1b0 0x7f30fc0cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f30fad9d700 1 -- 172.21.15.12:0/1171235815 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc0ceac0 con 0x7f30fc15d6c0 2023-07-08T12:38:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.046+0000 7f30fad9d700 1 --2- 172.21.15.12:0/1171235815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30fc15d6c0 0x7f30fc071530 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f30fc070b90 tx=0x7f30f400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6c9c7b77792d6d server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.047+0000 7f30f959a700 1 -- 172.21.15.12:0/1171235815 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4010070 con 0x7f30fc15d6c0 2023-07-08T12:38:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.047+0000 7f30f959a700 1 -- 172.21.15.12:0/1171235815 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f400dce0 con 0x7f30fc15d6c0 2023-07-08T12:38:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.047+0000 7f30f959a700 1 -- 172.21.15.12:0/1171235815 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f400b310 con 0x7f30fc15d6c0 2023-07-08T12:38:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.047+0000 7f3100827700 1 -- 172.21.15.12:0/1171235815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30fc15d6c0 msgr2=0x7f30fc071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.047+0000 7f3100827700 1 --2- 172.21.15.12:0/1171235815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30fc15d6c0 0x7f30fc071530 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f30fc070b90 tx=0x7f30f400d430 comp rx=0 tx=0).stop 2023-07-08T12:38:21.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.047+0000 7f3100827700 1 -- 172.21.15.12:0/1171235815 shutdown_connections 2023-07-08T12:38:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.047+0000 7f3100827700 1 --2- 172.21.15.12:0/1171235815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30fc15d6c0 0x7f30fc071530 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.047+0000 7f3100827700 1 --2- 172.21.15.12:0/1171235815 >> v2:172.21.15.160:3300/0 conn(0x7f30fc0ce1b0 0x7f30fc0cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.047+0000 7f3100827700 1 --2- 172.21.15.12:0/1171235815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30fc0b1c70 0x7f30fc0ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.047+0000 7f3100827700 1 -- 172.21.15.12:0/1171235815 >> 172.21.15.12:0/1171235815 conn(0x7f30fc0cc290 msgr2=0x7f30fc0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.047+0000 7f3100827700 1 -- 172.21.15.12:0/1171235815 shutdown_connections 2023-07-08T12:38:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.047+0000 7f3100827700 1 -- 172.21.15.12:0/1171235815 wait complete. 2023-07-08T12:38:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.049+0000 7f3100827700 1 Processor -- start 2023-07-08T12:38:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.049+0000 7f3100827700 1 -- start start 2023-07-08T12:38:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.049+0000 7f3100827700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30fc0b1c70 0x7f30fc0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.049+0000 7f3100827700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30fc0ce1b0 0x7f30fc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.049+0000 7f3100827700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30fc15d6c0 0x7f30fc14caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.049+0000 7f3100827700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0b4920 con 0x7f30fc0b1c70 2023-07-08T12:38:21.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.049+0000 7f3100827700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0aba50 con 0x7f30fc15d6c0 2023-07-08T12:38:21.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.049+0000 7f3100827700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0abcf0 con 0x7f30fc0ce1b0 2023-07-08T12:38:21.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30fa59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30fc0b1c70 0x7f30fc0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30fa59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30fc0b1c70 0x7f30fc0c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49354/0 (socket says 172.21.15.12:49354) 2023-07-08T12:38:21.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30fa59c700 1 -- 172.21.15.12:0/1152271325 learned_addr learned my addr 172.21.15.12:0/1152271325 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:21.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30fad9d700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30fc15d6c0 0x7f30fc14caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30f9d9b700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30fc0ce1b0 0x7f30fc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30fa59c700 1 -- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30fc0ce1b0 msgr2=0x7f30fc0b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30fa59c700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30fc0ce1b0 0x7f30fc0b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30fa59c700 1 -- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30fc15d6c0 msgr2=0x7f30fc14caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30fa59c700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30fc15d6c0 0x7f30fc14caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30fa59c700 1 -- 172.21.15.12:0/1152271325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f400bd40 con 0x7f30fc0b1c70 2023-07-08T12:38:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30fad9d700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30fc15d6c0 0x7f30fc14caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30f9d9b700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30fc0ce1b0 0x7f30fc0b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30fa59c700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30fc0b1c70 0x7f30fc0c92b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00e870 tx=0x7f30ec00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:21.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30eaffd700 1 -- 172.21.15.12:0/1152271325 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec009590 con 0x7f30fc0b1c70 2023-07-08T12:38:21.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30eaffd700 1 -- 172.21.15.12:0/1152271325 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ec00ee70 con 0x7f30fc0b1c70 2023-07-08T12:38:21.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f3100827700 1 -- 172.21.15.12:0/1152271325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30fc0abfd0 con 0x7f30fc0b1c70 2023-07-08T12:38:21.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f30eaffd700 1 -- 172.21.15.12:0/1152271325 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec00ca50 con 0x7f30fc0b1c70 2023-07-08T12:38:21.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.050+0000 7f3100827700 1 -- 172.21.15.12:0/1152271325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc0c97f0 con 0x7f30fc0b1c70 2023-07-08T12:38:21.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.051+0000 7f3100827700 1 -- 172.21.15.12:0/1152271325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30fc0cef20 con 0x7f30fc0b1c70 2023-07-08T12:38:21.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.052+0000 7f30eaffd700 1 -- 172.21.15.12:0/1152271325 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f30ec00cbb0 con 0x7f30fc0b1c70 2023-07-08T12:38:21.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.052+0000 7f30eaffd700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f30dc06ace0 0x7f30dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.052+0000 7f30eaffd700 1 -- 172.21.15.12:0/1152271325 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f30ec0896a0 con 0x7f30fc0b1c70 2023-07-08T12:38:21.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.052+0000 7f30f9d9b700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f30dc06ace0 0x7f30dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.053+0000 7f30f9d9b700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f30dc06ace0 0x7f30dc06d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009a40 tx=0x7f30f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:21.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.056+0000 7f30eaffd700 1 -- 172.21.15.12:0/1152271325 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f30ec057f80 con 0x7f30fc0b1c70 2023-07-08T12:38:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.271+0000 7f3100827700 1 -- 172.21.15.12:0/1152271325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f30fc0b4c00 con 0x7f30fc0b1c70 2023-07-08T12:38:21.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.271+0000 7f30eaffd700 1 -- 172.21.15.12:0/1152271325 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f30ec00ce60 con 0x7f30fc0b1c70 2023-07-08T12:38:21.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.273+0000 7f3100827700 1 -- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f30dc06ace0 msgr2=0x7f30dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.273+0000 7f3100827700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f30dc06ace0 0x7f30dc06d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009a40 tx=0x7f30f0008040 comp rx=0 tx=0).stop 2023-07-08T12:38:21.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.273+0000 7f3100827700 1 -- 172.21.15.12:0/1152271325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30fc0b1c70 msgr2=0x7f30fc0c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.273+0000 7f3100827700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30fc0b1c70 0x7f30fc0c92b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00e870 tx=0x7f30ec00ec30 comp rx=0 tx=0).stop 2023-07-08T12:38:21.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.273+0000 7f3100827700 1 -- 172.21.15.12:0/1152271325 shutdown_connections 2023-07-08T12:38:21.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.273+0000 7f3100827700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f30dc06ace0 0x7f30dc06d190 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.273+0000 7f3100827700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30fc15d6c0 0x7f30fc14caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.273+0000 7f3100827700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30fc0b1c70 0x7f30fc0c92b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.273+0000 7f3100827700 1 --2- 172.21.15.12:0/1152271325 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30fc0ce1b0 0x7f30fc0b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.273+0000 7f3100827700 1 -- 172.21.15.12:0/1152271325 >> 172.21.15.12:0/1152271325 conn(0x7f30fc0cc290 msgr2=0x7f30fc071430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.273+0000 7f3100827700 1 -- 172.21.15.12:0/1152271325 shutdown_connections 2023-07-08T12:38:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.273+0000 7f3100827700 1 -- 172.21.15.12:0/1152271325 wait complete. 2023-07-08T12:38:21.278 INFO:teuthology.orchestra.run.smithi012.stderr:listed 0 entries 2023-07-08T12:38:21.289 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:38:21.289 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:38:21.306 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:21.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.468+0000 7f2d0ed4c700 1 Processor -- start 2023-07-08T12:38:21.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.468+0000 7f2d0ed4c700 1 -- start start 2023-07-08T12:38:21.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d0ed4c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080cd530 0x7f2d0815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d0ed4c700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2d0806b640 0x7f2d080c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d0ed4c700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0806b1b0 con 0x7f2d08070bb0 2023-07-08T12:38:21.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d0ed4c700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0815d1f0 con 0x7f2d0806b640 2023-07-08T12:38:21.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d0ed4c700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0806a5a0 con 0x7f2d080cd530 2023-07-08T12:38:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d0cae8700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2d08070bb0 0x7f2d08070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36758/0 (socket says 172.21.15.12:36758) 2023-07-08T12:38:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d0cae8700 1 -- 172.21.15.12:0/3750960237 learned_addr learned my addr 172.21.15.12:0/3750960237 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d0d2e9700 1 --2- 172.21.15.12:0/3750960237 >> v2:172.21.15.160:3300/0 conn(0x7f2d0806b640 0x7f2d080c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d07fff700 1 --2- 172.21.15.12:0/3750960237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080cd530 0x7f2d0815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567530038 0 0) 0x7f2d0806b1b0 con 0x7f2d08070bb0 2023-07-08T12:38:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf80035b0 con 0x7f2d08070bb0 2023-07-08T12:38:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1970622627 0 0) 0x7f2cf80035b0 con 0x7f2d08070bb0 2023-07-08T12:38:21.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080cd530 msgr2=0x7f2d0815ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:21.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 --2- 172.21.15.12:0/3750960237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080cd530 0x7f2d0815ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 >> v2:172.21.15.160:3300/0 conn(0x7f2d0806b640 msgr2=0x7f2d080c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:21.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 --2- 172.21.15.12:0/3750960237 >> v2:172.21.15.160:3300/0 conn(0x7f2d0806b640 0x7f2d080c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0806a880 con 0x7f2d08070bb0 2023-07-08T12:38:21.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f2cfc001670 con 0x7f2d08070bb0 2023-07-08T12:38:21.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 >> v1:172.21.15.12:6789/0 conn(0x7f2d08070bb0 legacy=0x7f2d08070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 --2- 172.21.15.12:0/3750960237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cf8003a00 0x7f2cf8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 --2- 172.21.15.12:0/3750960237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf8004430 0x7f2cf80048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 --2- 172.21.15.12:0/3750960237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080cd530 0x7f2cf8004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0815d1f0 con 0x7f2cf8004430 2023-07-08T12:38:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.469+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0806a5a0 con 0x7f2cf8003a00 2023-07-08T12:38:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf800ba10 con 0x7f2d080cd530 2023-07-08T12:38:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d07fff700 1 --2- 172.21.15.12:0/3750960237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf8004430 0x7f2cf80048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d0cae8700 1 --2- 172.21.15.12:0/3750960237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cf8003a00 0x7f2cf8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d0d2e9700 1 --2- 172.21.15.12:0/3750960237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080cd530 0x7f2cf8004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d07fff700 1 -- 172.21.15.12:0/3750960237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080cd530 msgr2=0x7f2cf8004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:21.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d07fff700 1 --2- 172.21.15.12:0/3750960237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080cd530 0x7f2cf8004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d07fff700 1 -- 172.21.15.12:0/3750960237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cf8003a00 msgr2=0x7f2cf8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:21.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d07fff700 1 --2- 172.21.15.12:0/3750960237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cf8003a00 0x7f2cf8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d07fff700 1 -- 172.21.15.12:0/3750960237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf80035b0 con 0x7f2cf8004430 2023-07-08T12:38:21.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d0cae8700 1 --2- 172.21.15.12:0/3750960237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cf8003a00 0x7f2cf8003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:21.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d07fff700 1 --2- 172.21.15.12:0/3750960237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf8004430 0x7f2cf80048f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009b60 tx=0x7f2cf8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b731b8f36617853 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400e6c0 con 0x7f2cf8004430 2023-07-08T12:38:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf400e820 con 0x7f2cf8004430 2023-07-08T12:38:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d077fe700 1 -- 172.21.15.12:0/3750960237 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400eb90 con 0x7f2cf8004430 2023-07-08T12:38:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.470+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/3750960237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf8004430 msgr2=0x7f2cf80048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.471+0000 7f2d0ed4c700 1 --2- 172.21.15.12:0/3750960237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cf8004430 0x7f2cf80048f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009b60 tx=0x7f2cf8005880 comp rx=0 tx=0).stop 2023-07-08T12:38:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.471+0000 7f2d0d2e9700 1 -- 172.21.15.12:0/3750960237 reap_dead start 2023-07-08T12:38:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.471+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/3750960237 shutdown_connections 2023-07-08T12:38:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.471+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/3750960237 >> 172.21.15.12:0/3750960237 conn(0x7f2d080cb9f0 msgr2=0x7f2d080cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.471+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/3750960237 shutdown_connections 2023-07-08T12:38:21.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.471+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/3750960237 wait complete. 2023-07-08T12:38:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.472+0000 7f2d0ed4c700 1 Processor -- start 2023-07-08T12:38:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.472+0000 7f2d0ed4c700 1 -- start start 2023-07-08T12:38:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.473+0000 7f2d0ed4c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cf8004430 0x7f2d0815cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.473+0000 7f2d0ed4c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d081505b0 0x7f2d0814c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.473+0000 7f2d0ed4c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d08156fa0 0x7f2d080ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.473+0000 7f2d0ed4c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0815d070 con 0x7f2d081505b0 2023-07-08T12:38:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.473+0000 7f2d0ed4c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08156b10 con 0x7f2cf8004430 2023-07-08T12:38:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.473+0000 7f2d0ed4c700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08158050 con 0x7f2d08156fa0 2023-07-08T12:38:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.473+0000 7f2d07fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d081505b0 0x7f2d0814c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d07fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d081505b0 0x7f2d0814c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49376/0 (socket says 172.21.15.12:49376) 2023-07-08T12:38:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d07fff700 1 -- 172.21.15.12:0/516901261 learned_addr learned my addr 172.21.15.12:0/516901261 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d0cae8700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cf8004430 0x7f2d0815cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d0d2e9700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d08156fa0 0x7f2d080ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d07fff700 1 -- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d08156fa0 msgr2=0x7f2d080ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d07fff700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d08156fa0 0x7f2d080ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d07fff700 1 -- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cf8004430 msgr2=0x7f2d0815cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d07fff700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cf8004430 0x7f2d0815cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d07fff700 1 -- 172.21.15.12:0/516901261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d08158330 con 0x7f2d081505b0 2023-07-08T12:38:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d0cae8700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cf8004430 0x7f2d0815cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d0d2e9700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d08156fa0 0x7f2d080ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d07fff700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d081505b0 0x7f2d0814c770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8005760 tx=0x7f2cf40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:21.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d057fa700 1 -- 172.21.15.12:0/516901261 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400e6c0 con 0x7f2d081505b0 2023-07-08T12:38:21.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d057fa700 1 -- 172.21.15.12:0/516901261 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf4003ad0 con 0x7f2d081505b0 2023-07-08T12:38:21.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/516901261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d08158610 con 0x7f2d081505b0 2023-07-08T12:38:21.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d057fa700 1 -- 172.21.15.12:0/516901261 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4003e80 con 0x7f2d081505b0 2023-07-08T12:38:21.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.474+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/516901261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d081567a0 con 0x7f2d081505b0 2023-07-08T12:38:21.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.475+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/516901261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d08156320 con 0x7f2d081505b0 2023-07-08T12:38:21.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.476+0000 7f2d057fa700 1 -- 172.21.15.12:0/516901261 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2cf4013470 con 0x7f2d081505b0 2023-07-08T12:38:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.476+0000 7f2d057fa700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2cec06ace0 0x7f2cec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.476+0000 7f2d057fa700 1 -- 172.21.15.12:0/516901261 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2cf408b060 con 0x7f2d081505b0 2023-07-08T12:38:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.476+0000 7f2d0cae8700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2cec06ace0 0x7f2cec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.477+0000 7f2d0cae8700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2cec06ace0 0x7f2cec06d190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc000fb0 tx=0x7f2cfc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.480+0000 7f2d057fa700 1 -- 172.21.15.12:0/516901261 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2d08156320 con 0x7f2d081505b0 2023-07-08T12:38:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.678+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/516901261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d08156320 con 0x7f2d081505b0 2023-07-08T12:38:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.679+0000 7f2d057fa700 1 -- 172.21.15.12:0/516901261 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f2d08156320 con 0x7f2d081505b0 2023-07-08T12:38:21.682 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:21.682 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:12.993803+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.681+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2cec06ace0 msgr2=0x7f2cec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.681+0000 7f2d0ed4c700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2cec06ace0 0x7f2cec06d190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc000fb0 tx=0x7f2cfc009040 comp rx=0 tx=0).stop 2023-07-08T12:38:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.681+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/516901261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d081505b0 msgr2=0x7f2d0814c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.681+0000 7f2d0ed4c700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d081505b0 0x7f2d0814c770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8005760 tx=0x7f2cf40047c0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.681+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/516901261 shutdown_connections 2023-07-08T12:38:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.681+0000 7f2d0ed4c700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2cec06ace0 0x7f2cec06d190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.681+0000 7f2d0ed4c700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d08156fa0 0x7f2d080ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.681+0000 7f2d0ed4c700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cf8004430 0x7f2d0815cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.681+0000 7f2d0ed4c700 1 --2- 172.21.15.12:0/516901261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d081505b0 0x7f2d0814c770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.681+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/516901261 >> 172.21.15.12:0/516901261 conn(0x7f2d080cb9f0 msgr2=0x7f2d080b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.681+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/516901261 shutdown_connections 2023-07-08T12:38:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.681+0000 7f2d0ed4c700 1 -- 172.21.15.12:0/516901261 wait complete. 2023-07-08T12:38:21.696 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-08T12:38:05.660194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-08T12:38:06.920177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-08T12:38:10.592749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-08T12:38:21.697 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:38:21.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.822+0000 7fbf1f9ac700 1 Processor -- start 2023-07-08T12:38:21.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.822+0000 7fbf1f9ac700 1 -- start start 2023-07-08T12:38:21.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.823+0000 7fbf1f9ac700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbf180aeec0 0x7fbf1806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.823+0000 7fbf1f9ac700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf1806ebe0 0x7fbf180cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.823+0000 7fbf1f9ac700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbf180cd9b0 0x7fbf180cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.823+0000 7fbf1f9ac700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1815d770 con 0x7fbf180aeec0 2023-07-08T12:38:21.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.823+0000 7fbf1f9ac700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf180cddd0 con 0x7fbf180cd9b0 2023-07-08T12:38:21.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.823+0000 7fbf1f9ac700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1815cb60 con 0x7fbf1806ebe0 2023-07-08T12:38:21.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.823+0000 7fbf1d748700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbf180aeec0 0x7fbf1806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.823+0000 7fbf1d748700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbf180aeec0 0x7fbf1806cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49382/0 (socket says 172.21.15.12:49382) 2023-07-08T12:38:21.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.823+0000 7fbf1d748700 1 -- 172.21.15.12:0/3581837978 learned_addr learned my addr 172.21.15.12:0/3581837978 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:21.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.823+0000 7fbf1df49700 1 --2- 172.21.15.12:0/3581837978 >> v2:172.21.15.160:3300/0 conn(0x7fbf180cd9b0 0x7fbf180cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.823+0000 7fbf1cf47700 1 --2- 172.21.15.12:0/3581837978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf1806ebe0 0x7fbf180cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf1d748700 1 -- 172.21.15.12:0/3581837978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf1806ebe0 msgr2=0x7fbf180cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:21.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf1d748700 1 --2- 172.21.15.12:0/3581837978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf1806ebe0 0x7fbf180cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf1d748700 1 -- 172.21.15.12:0/3581837978 >> v2:172.21.15.160:3300/0 conn(0x7fbf180cd9b0 msgr2=0x7fbf180cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:21.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf1d748700 1 --2- 172.21.15.12:0/3581837978 >> v2:172.21.15.160:3300/0 conn(0x7fbf180cd9b0 0x7fbf180cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf1d748700 1 -- 172.21.15.12:0/3581837978 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1815ce40 con 0x7fbf180aeec0 2023-07-08T12:38:21.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf1d748700 1 --2- 172.21.15.12:0/3581837978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf180aeec0 0x7fbf1806cff0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbf180724b0 tx=0x7fbf14009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2592312eab6dacc7 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:21.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf0ffff700 1 -- 172.21.15.12:0/3581837978 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14005e30 con 0x7fbf180aeec0 2023-07-08T12:38:21.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf0ffff700 1 -- 172.21.15.12:0/3581837978 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf14004030 con 0x7fbf180aeec0 2023-07-08T12:38:21.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf0ffff700 1 -- 172.21.15.12:0/3581837978 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf140043e0 con 0x7fbf180aeec0 2023-07-08T12:38:21.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/3581837978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf180aeec0 msgr2=0x7fbf1806cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf1f9ac700 1 --2- 172.21.15.12:0/3581837978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf180aeec0 0x7fbf1806cff0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbf180724b0 tx=0x7fbf14009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/3581837978 shutdown_connections 2023-07-08T12:38:21.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf1f9ac700 1 --2- 172.21.15.12:0/3581837978 >> v2:172.21.15.160:3300/0 conn(0x7fbf180cd9b0 0x7fbf180cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf1f9ac700 1 --2- 172.21.15.12:0/3581837978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf180aeec0 0x7fbf1806cff0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf1f9ac700 1 --2- 172.21.15.12:0/3581837978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf1806ebe0 0x7fbf180cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.824+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/3581837978 >> 172.21.15.12:0/3581837978 conn(0x7fbf18098b50 msgr2=0x7fbf1806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:21.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.825+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/3581837978 shutdown_connections 2023-07-08T12:38:21.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.825+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/3581837978 wait complete. 2023-07-08T12:38:21.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.826+0000 7fbf1f9ac700 1 Processor -- start 2023-07-08T12:38:21.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.826+0000 7fbf1f9ac700 1 -- start start 2023-07-08T12:38:21.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.826+0000 7fbf1f9ac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf1806ebe0 0x7fbf180cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.826+0000 7fbf1f9ac700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf180aeec0 0x7fbf180c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.826+0000 7fbf1f9ac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf180cd9b0 0x7fbf18154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.826+0000 7fbf1f9ac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180c93c0 con 0x7fbf1806ebe0 2023-07-08T12:38:21.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.826+0000 7fbf1f9ac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180c8780 con 0x7fbf180cd9b0 2023-07-08T12:38:21.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.826+0000 7fbf1f9ac700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180c72c0 con 0x7fbf180aeec0 2023-07-08T12:38:21.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf1d748700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf1806ebe0 0x7fbf180cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf1d748700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf1806ebe0 0x7fbf180cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49388/0 (socket says 172.21.15.12:49388) 2023-07-08T12:38:21.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf1d748700 1 -- 172.21.15.12:0/2422058939 learned_addr learned my addr 172.21.15.12:0/2422058939 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:21.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf1df49700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf180cd9b0 0x7fbf18154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf1cf47700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf180aeec0 0x7fbf180c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf1d748700 1 -- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf180aeec0 msgr2=0x7fbf180c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf1d748700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf180aeec0 0x7fbf180c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf1d748700 1 -- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf180cd9b0 msgr2=0x7fbf18154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:21.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf1d748700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf180cd9b0 0x7fbf18154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:21.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf1d748700 1 -- 172.21.15.12:0/2422058939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf14009500 con 0x7fbf1806ebe0 2023-07-08T12:38:21.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf1d748700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf1806ebe0 0x7fbf180cfa10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009130 tx=0x7fbf14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:21.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf0dffb700 1 -- 172.21.15.12:0/2422058939 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14003dd0 con 0x7fbf1806ebe0 2023-07-08T12:38:21.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf0dffb700 1 -- 172.21.15.12:0/2422058939 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf14011420 con 0x7fbf1806ebe0 2023-07-08T12:38:21.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/2422058939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf180c8a20 con 0x7fbf1806ebe0 2023-07-08T12:38:21.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.827+0000 7fbf0dffb700 1 -- 172.21.15.12:0/2422058939 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf140117d0 con 0x7fbf1806ebe0 2023-07-08T12:38:21.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.828+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/2422058939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf180c8fe0 con 0x7fbf1806ebe0 2023-07-08T12:38:21.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.828+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/2422058939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf180c9500 con 0x7fbf1806ebe0 2023-07-08T12:38:21.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.829+0000 7fbf0dffb700 1 -- 172.21.15.12:0/2422058939 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf14011930 con 0x7fbf1806ebe0 2023-07-08T12:38:21.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.829+0000 7fbf0dffb700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbefc06ace0 0x7fbefc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:21.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.830+0000 7fbf0dffb700 1 -- 172.21.15.12:0/2422058939 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbf1408a9d0 con 0x7fbf1806ebe0 2023-07-08T12:38:21.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.830+0000 7fbf1cf47700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbefc06ace0 0x7fbefc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:21.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.830+0000 7fbf1cf47700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbefc06ace0 0x7fbefc06d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf18155cc0 tx=0x7fbf08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:21.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:21.834+0000 7fbf0dffb700 1 -- 172.21.15.12:0/2422058939 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbf14055820 con 0x7fbf1806ebe0 2023-07-08T12:38:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.030+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/2422058939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf18066af0 con 0x7fbf1806ebe0 2023-07-08T12:38:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.030+0000 7fbf0dffb700 1 -- 172.21.15.12:0/2422058939 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf14058e40 con 0x7fbf1806ebe0 2023-07-08T12:38:22.033 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:38:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.032+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbefc06ace0 msgr2=0x7fbefc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.032+0000 7fbf1f9ac700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbefc06ace0 0x7fbefc06d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf18155cc0 tx=0x7fbf08008040 comp rx=0 tx=0).stop 2023-07-08T12:38:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.032+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/2422058939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf1806ebe0 msgr2=0x7fbf180cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.032+0000 7fbf1f9ac700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf1806ebe0 0x7fbf180cfa10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009130 tx=0x7fbf14004750 comp rx=0 tx=0).stop 2023-07-08T12:38:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.032+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/2422058939 shutdown_connections 2023-07-08T12:38:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.032+0000 7fbf1f9ac700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbefc06ace0 0x7fbefc06d190 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.032+0000 7fbf1f9ac700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf180cd9b0 0x7fbf18154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.032+0000 7fbf1f9ac700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf1806ebe0 0x7fbf180cfa10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.032+0000 7fbf1f9ac700 1 --2- 172.21.15.12:0/2422058939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf180aeec0 0x7fbf180c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.032+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/2422058939 >> 172.21.15.12:0/2422058939 conn(0x7fbf18098b50 msgr2=0x7fbf180ad7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.032+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/2422058939 shutdown_connections 2023-07-08T12:38:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.033+0000 7fbf1f9ac700 1 -- 172.21.15.12:0/2422058939 wait complete. 2023-07-08T12:38:22.047 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:38:22.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.172+0000 7fe9a879c700 1 Processor -- start 2023-07-08T12:38:22.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.172+0000 7fe9a879c700 1 -- start start 2023-07-08T12:38:22.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.173+0000 7fe9a879c700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe9a0156a10 0x7fe9a006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.173+0000 7fe9a879c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe9a015d390 0x7fe9a01608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.173+0000 7fe9a879c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9a00cc240 0x7fe9a00cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.173+0000 7fe9a879c700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a00ccc00 con 0x7fe9a015d390 2023-07-08T12:38:22.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.173+0000 7fe9a879c700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a0160410 con 0x7fe9a0156a10 2023-07-08T12:38:22.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.173+0000 7fe9a879c700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a0156de0 con 0x7fe9a00cc240 2023-07-08T12:38:22.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.173+0000 7fe9a5d37700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe9a015d390 0x7fe9a01608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.173+0000 7fe9a5d37700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe9a015d390 0x7fe9a01608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49398/0 (socket says 172.21.15.12:49398) 2023-07-08T12:38:22.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.173+0000 7fe9a5d37700 1 -- 172.21.15.12:0/2077151397 learned_addr learned my addr 172.21.15.12:0/2077151397 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:22.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.173+0000 7fe9a6538700 1 --2- 172.21.15.12:0/2077151397 >> v2:172.21.15.160:3300/0 conn(0x7fe9a0156a10 0x7fe9a006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.173+0000 7fe9a6d39700 1 --2- 172.21.15.12:0/2077151397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9a00cc240 0x7fe9a00cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.174+0000 7fe9a5d37700 1 -- 172.21.15.12:0/2077151397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9a00cc240 msgr2=0x7fe9a00cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:22.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.174+0000 7fe9a5d37700 1 --2- 172.21.15.12:0/2077151397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9a00cc240 0x7fe9a00cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.174+0000 7fe9a5d37700 1 -- 172.21.15.12:0/2077151397 >> v2:172.21.15.160:3300/0 conn(0x7fe9a0156a10 msgr2=0x7fe9a006e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:22.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.174+0000 7fe9a5d37700 1 --2- 172.21.15.12:0/2077151397 >> v2:172.21.15.160:3300/0 conn(0x7fe9a0156a10 0x7fe9a006e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.174+0000 7fe9a5d37700 1 -- 172.21.15.12:0/2077151397 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a006ec70 con 0x7fe9a015d390 2023-07-08T12:38:22.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.174+0000 7fe9a6d39700 1 --2- 172.21.15.12:0/2077151397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9a00cc240 0x7fe9a00cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:22.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.174+0000 7fe9a5d37700 1 --2- 172.21.15.12:0/2077151397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a015d390 0x7fe9a01608a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0072490 tx=0x7fe99c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768010c5c99bed08 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:22.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.174+0000 7fe9a5536700 1 -- 172.21.15.12:0/2077151397 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c011830 con 0x7fe9a015d390 2023-07-08T12:38:22.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.174+0000 7fe9a5536700 1 -- 172.21.15.12:0/2077151397 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99c011990 con 0x7fe9a015d390 2023-07-08T12:38:22.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.174+0000 7fe9a5536700 1 -- 172.21.15.12:0/2077151397 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c011d20 con 0x7fe9a015d390 2023-07-08T12:38:22.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.174+0000 7fe9a879c700 1 -- 172.21.15.12:0/2077151397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a015d390 msgr2=0x7fe9a01608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.174+0000 7fe9a879c700 1 --2- 172.21.15.12:0/2077151397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a015d390 0x7fe9a01608a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0072490 tx=0x7fe99c009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.175+0000 7fe9a879c700 1 -- 172.21.15.12:0/2077151397 shutdown_connections 2023-07-08T12:38:22.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.175+0000 7fe9a879c700 1 --2- 172.21.15.12:0/2077151397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9a00cc240 0x7fe9a00cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.175+0000 7fe9a879c700 1 --2- 172.21.15.12:0/2077151397 >> v2:172.21.15.160:3300/0 conn(0x7fe9a0156a10 0x7fe9a006e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.175+0000 7fe9a879c700 1 --2- 172.21.15.12:0/2077151397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a015d390 0x7fe9a01608a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.175+0000 7fe9a879c700 1 -- 172.21.15.12:0/2077151397 >> 172.21.15.12:0/2077151397 conn(0x7fe9a0098b50 msgr2=0x7fe9a006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:22.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.175+0000 7fe9a879c700 1 -- 172.21.15.12:0/2077151397 shutdown_connections 2023-07-08T12:38:22.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.175+0000 7fe9a879c700 1 -- 172.21.15.12:0/2077151397 wait complete. 2023-07-08T12:38:22.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.176+0000 7fe9a879c700 1 Processor -- start 2023-07-08T12:38:22.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.176+0000 7fe9a879c700 1 -- start start 2023-07-08T12:38:22.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a879c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a00cc240 0x7fe9a01f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a879c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a0156a10 0x7fe9a01597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a879c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9a015d390 0x7fe9a01558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a879c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00b34d0 con 0x7fe9a00cc240 2023-07-08T12:38:22.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a879c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00b3720 con 0x7fe9a0156a10 2023-07-08T12:38:22.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a879c700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a00b3a00 con 0x7fe9a015d390 2023-07-08T12:38:22.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a6538700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a00cc240 0x7fe9a01f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a6538700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a00cc240 0x7fe9a01f2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49402/0 (socket says 172.21.15.12:49402) 2023-07-08T12:38:22.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a6538700 1 -- 172.21.15.12:0/3730495421 learned_addr learned my addr 172.21.15.12:0/3730495421 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:22.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a5d37700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a0156a10 0x7fe9a01597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a6d39700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9a015d390 0x7fe9a01558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a6538700 1 -- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9a015d390 msgr2=0x7fe9a01558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a6538700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9a015d390 0x7fe9a01558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a6538700 1 -- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a0156a10 msgr2=0x7fe9a01597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a6538700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a0156a10 0x7fe9a01597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a6538700 1 -- 172.21.15.12:0/3730495421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c009550 con 0x7fe9a00cc240 2023-07-08T12:38:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a6538700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a00cc240 0x7fe9a01f2160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe99000e990 tx=0x7fe99000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe996ffd700 1 -- 172.21.15.12:0/3730495421 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9900096f0 con 0x7fe9a00cc240 2023-07-08T12:38:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe996ffd700 1 -- 172.21.15.12:0/3730495421 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99000c850 con 0x7fe9a00cc240 2023-07-08T12:38:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe996ffd700 1 -- 172.21.15.12:0/3730495421 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99000cbc0 con 0x7fe9a00cc240 2023-07-08T12:38:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a879c700 1 -- 172.21.15.12:0/3730495421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a00b3ce0 con 0x7fe9a00cc240 2023-07-08T12:38:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.177+0000 7fe9a879c700 1 -- 172.21.15.12:0/3730495421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a00b42a0 con 0x7fe9a00cc240 2023-07-08T12:38:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.178+0000 7fe9a879c700 1 -- 172.21.15.12:0/3730495421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a00b4460 con 0x7fe9a00cc240 2023-07-08T12:38:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.179+0000 7fe996ffd700 1 -- 172.21.15.12:0/3730495421 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe99000cd20 con 0x7fe9a00cc240 2023-07-08T12:38:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.179+0000 7fe996ffd700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe98406ace0 0x7fe98406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.179+0000 7fe996ffd700 1 -- 172.21.15.12:0/3730495421 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe990089bc0 con 0x7fe9a00cc240 2023-07-08T12:38:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.179+0000 7fe9a5d37700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe98406ace0 0x7fe98406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.180+0000 7fe9a5d37700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe98406ace0 0x7fe98406d190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe99c014040 tx=0x7fe99c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.184+0000 7fe996ffd700 1 -- 172.21.15.12:0/3730495421 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe990054a10 con 0x7fe9a00cc240 2023-07-08T12:38:22.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.385+0000 7fe9a879c700 1 -- 172.21.15.12:0/3730495421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe9a00b4610 con 0x7fe9a00cc240 2023-07-08T12:38:22.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.385+0000 7fe996ffd700 1 -- 172.21.15.12:0/3730495421 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe9900170a0 con 0x7fe9a00cc240 2023-07-08T12:38:22.387 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:38:22.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.387+0000 7fe9a879c700 1 -- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe98406ace0 msgr2=0x7fe98406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.387+0000 7fe9a879c700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe98406ace0 0x7fe98406d190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe99c014040 tx=0x7fe99c008030 comp rx=0 tx=0).stop 2023-07-08T12:38:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.387+0000 7fe9a879c700 1 -- 172.21.15.12:0/3730495421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a00cc240 msgr2=0x7fe9a01f2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.387+0000 7fe9a879c700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a00cc240 0x7fe9a01f2160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe99000e990 tx=0x7fe99000ed50 comp rx=0 tx=0).stop 2023-07-08T12:38:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.387+0000 7fe9a879c700 1 -- 172.21.15.12:0/3730495421 shutdown_connections 2023-07-08T12:38:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.387+0000 7fe9a879c700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe98406ace0 0x7fe98406d190 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.387+0000 7fe9a879c700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9a015d390 0x7fe9a01558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.387+0000 7fe9a879c700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a00cc240 0x7fe9a01f2160 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.387+0000 7fe9a879c700 1 --2- 172.21.15.12:0/3730495421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a0156a10 0x7fe9a01597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.387+0000 7fe9a879c700 1 -- 172.21.15.12:0/3730495421 >> 172.21.15.12:0/3730495421 conn(0x7fe9a0098b50 msgr2=0x7fe9a00b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.387+0000 7fe9a879c700 1 -- 172.21.15.12:0/3730495421 shutdown_connections 2023-07-08T12:38:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.387+0000 7fe9a879c700 1 -- 172.21.15.12:0/3730495421 wait complete. 2023-07-08T12:38:22.401 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-08T12:38:22.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.527+0000 7f9e12a92700 1 Processor -- start 2023-07-08T12:38:22.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.527+0000 7f9e12a92700 1 -- start start 2023-07-08T12:38:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.528+0000 7f9e12a92700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9e0c15a190 0x7f9e0c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.528+0000 7f9e12a92700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9e0c0b12b0 0x7f9e0c0ac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.528+0000 7f9e12a92700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0c06d2b0 con 0x7f9e0c0ac960 2023-07-08T12:38:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.528+0000 7f9e12a92700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0c15a560 con 0x7f9e0c0b12b0 2023-07-08T12:38:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.528+0000 7f9e12a92700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0c06d590 con 0x7f9e0c15a190 2023-07-08T12:38:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.528+0000 7f9e0bfff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9e0c0b12b0 0x7f9e0c0ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.528+0000 7f9e1102f700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9e0c0ac960 0x7f9e0c069f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36810/0 (socket says 172.21.15.12:36810) 2023-07-08T12:38:22.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.528+0000 7f9e0bfff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9e0c0b12b0 0x7f9e0c0ac420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49412/0 (socket says 172.21.15.12:49412) 2023-07-08T12:38:22.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.528+0000 7f9e1102f700 1 -- 172.21.15.12:0/3620259336 learned_addr learned my addr 172.21.15.12:0/3620259336 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:22.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.528+0000 7f9e1082e700 1 --2- 172.21.15.12:0/3620259336 >> v2:172.21.15.160:3300/0 conn(0x7f9e0c15a190 0x7f9e0c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.528+0000 7f9e0bfff700 1 -- 172.21.15.12:0/3620259336 >> v2:172.21.15.160:3300/0 conn(0x7f9e0c15a190 msgr2=0x7f9e0c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:22.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.528+0000 7f9e0bfff700 1 --2- 172.21.15.12:0/3620259336 >> v2:172.21.15.160:3300/0 conn(0x7f9e0c15a190 0x7f9e0c06e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e0bfff700 1 -- 172.21.15.12:0/3620259336 >> v1:172.21.15.12:6789/0 conn(0x7f9e0c0ac960 legacy=0x7f9e0c069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e0bfff700 1 -- 172.21.15.12:0/3620259336 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c06d870 con 0x7f9e0c0b12b0 2023-07-08T12:38:22.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e0b7fe700 1 -- 172.21.15.12:0/3620259336 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378987834 0 0) 0x7f9e0c06d2b0 con 0x7f9e0c0ac960 2023-07-08T12:38:22.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e0bfff700 1 --2- 172.21.15.12:0/3620259336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e0c0b12b0 0x7f9e0c0ac420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c06c0d0 tx=0x7f9df8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e584be7145edf34 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:22.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e0b7fe700 1 -- 172.21.15.12:0/3620259336 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8011830 con 0x7f9e0c0b12b0 2023-07-08T12:38:22.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e0b7fe700 1 -- 172.21.15.12:0/3620259336 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df8011990 con 0x7f9e0c0b12b0 2023-07-08T12:38:22.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e0b7fe700 1 -- 172.21.15.12:0/3620259336 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8011d20 con 0x7f9e0c0b12b0 2023-07-08T12:38:22.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e12a92700 1 -- 172.21.15.12:0/3620259336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e0c0b12b0 msgr2=0x7f9e0c0ac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e12a92700 1 --2- 172.21.15.12:0/3620259336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e0c0b12b0 0x7f9e0c0ac420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c06c0d0 tx=0x7f9df8009b20 comp rx=0 tx=0).stop 2023-07-08T12:38:22.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e12a92700 1 -- 172.21.15.12:0/3620259336 shutdown_connections 2023-07-08T12:38:22.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e12a92700 1 --2- 172.21.15.12:0/3620259336 >> v2:172.21.15.160:3300/0 conn(0x7f9e0c15a190 0x7f9e0c06e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e12a92700 1 --2- 172.21.15.12:0/3620259336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e0c0b12b0 0x7f9e0c0ac420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e12a92700 1 -- 172.21.15.12:0/3620259336 >> 172.21.15.12:0/3620259336 conn(0x7f9e0c098b50 msgr2=0x7f9e0c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:22.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e12a92700 1 -- 172.21.15.12:0/3620259336 shutdown_connections 2023-07-08T12:38:22.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.529+0000 7f9e12a92700 1 -- 172.21.15.12:0/3620259336 wait complete. 2023-07-08T12:38:22.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.531+0000 7f9e12a92700 1 Processor -- start 2023-07-08T12:38:22.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.531+0000 7f9e12a92700 1 -- start start 2023-07-08T12:38:22.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e12a92700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9e0c0ac960 0x7f9e0c155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e12a92700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e0c0b12b0 0x7f9e0c0b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e12a92700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e0c15a190 0x7f9e0c0c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e12a92700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c0b5510 con 0x7f9e0c15a190 2023-07-08T12:38:22.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e12a92700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c0ce620 con 0x7f9e0c0b12b0 2023-07-08T12:38:22.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e12a92700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c1556b0 con 0x7f9e0c0ac960 2023-07-08T12:38:22.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e1102f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e0c15a190 0x7f9e0c0c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e1102f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e0c15a190 0x7f9e0c0c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49420/0 (socket says 172.21.15.12:49420) 2023-07-08T12:38:22.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e1102f700 1 -- 172.21.15.12:0/1971759143 learned_addr learned my addr 172.21.15.12:0/1971759143 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:22.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e0bfff700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e0c0b12b0 0x7f9e0c0b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e1082e700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9e0c0ac960 0x7f9e0c155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e1102f700 1 -- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9e0c0ac960 msgr2=0x7f9e0c155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e1102f700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9e0c0ac960 0x7f9e0c155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e1102f700 1 -- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e0c0b12b0 msgr2=0x7f9e0c0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e1102f700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e0c0b12b0 0x7f9e0c0b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e1102f700 1 -- 172.21.15.12:0/1971759143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df80094a0 con 0x7f9e0c15a190 2023-07-08T12:38:22.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e0bfff700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e0c0b12b0 0x7f9e0c0b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:22.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e1102f700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e0c15a190 0x7f9e0c0c73b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400cba0 tx=0x7f9e0400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:22.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e097fa700 1 -- 172.21.15.12:0/1971759143 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0400ab80 con 0x7f9e0c15a190 2023-07-08T12:38:22.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e097fa700 1 -- 172.21.15.12:0/1971759143 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0400ad00 con 0x7f9e0c15a190 2023-07-08T12:38:22.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e12a92700 1 -- 172.21.15.12:0/1971759143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0c0b4c70 con 0x7f9e0c15a190 2023-07-08T12:38:22.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e097fa700 1 -- 172.21.15.12:0/1971759143 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e04007310 con 0x7f9e0c15a190 2023-07-08T12:38:22.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.532+0000 7f9e12a92700 1 -- 172.21.15.12:0/1971759143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0c0b5230 con 0x7f9e0c15a190 2023-07-08T12:38:22.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.533+0000 7f9e12a92700 1 -- 172.21.15.12:0/1971759143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0c0ce760 con 0x7f9e0c15a190 2023-07-08T12:38:22.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.534+0000 7f9e097fa700 1 -- 172.21.15.12:0/1971759143 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9e04007470 con 0x7f9e0c15a190 2023-07-08T12:38:22.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.534+0000 7f9e097fa700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9df006ace0 0x7f9df006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.534+0000 7f9e097fa700 1 -- 172.21.15.12:0/1971759143 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9e040895d0 con 0x7f9e0c15a190 2023-07-08T12:38:22.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.534+0000 7f9e1082e700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9df006ace0 0x7f9df006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.535+0000 7f9e1082e700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9df006ace0 0x7f9df006d190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9e00006fd0 tx=0x7f9e00008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:22.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.539+0000 7f9e097fa700 1 -- 172.21.15.12:0/1971759143 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9e0c0ce760 con 0x7f9e0c15a190 2023-07-08T12:38:22.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.738+0000 7f9e12a92700 1 -- 172.21.15.12:0/1971759143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e0c0ce760 con 0x7f9e0c15a190 2023-07-08T12:38:22.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.738+0000 7f9e097fa700 1 -- 172.21.15.12:0/1971759143 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e0c0ce760 con 0x7f9e0c15a190 2023-07-08T12:38:22.742 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:38:22.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.740+0000 7f9e12a92700 1 -- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9df006ace0 msgr2=0x7f9df006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.740+0000 7f9e12a92700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9df006ace0 0x7f9df006d190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9e00006fd0 tx=0x7f9e00008040 comp rx=0 tx=0).stop 2023-07-08T12:38:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.740+0000 7f9e12a92700 1 -- 172.21.15.12:0/1971759143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e0c15a190 msgr2=0x7f9e0c0c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.740+0000 7f9e12a92700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e0c15a190 0x7f9e0c0c73b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400cba0 tx=0x7f9e0400cf60 comp rx=0 tx=0).stop 2023-07-08T12:38:22.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.740+0000 7f9e12a92700 1 -- 172.21.15.12:0/1971759143 shutdown_connections 2023-07-08T12:38:22.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.740+0000 7f9e12a92700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9df006ace0 0x7f9df006d190 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.740+0000 7f9e12a92700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e0c15a190 0x7f9e0c0c73b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.740+0000 7f9e12a92700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9e0c0ac960 0x7f9e0c155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.740+0000 7f9e12a92700 1 --2- 172.21.15.12:0/1971759143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e0c0b12b0 0x7f9e0c0b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.740+0000 7f9e12a92700 1 -- 172.21.15.12:0/1971759143 >> 172.21.15.12:0/1971759143 conn(0x7f9e0c098b50 msgr2=0x7f9e0c0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:22.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.740+0000 7f9e12a92700 1 -- 172.21.15.12:0/1971759143 shutdown_connections 2023-07-08T12:38:22.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.740+0000 7f9e12a92700 1 -- 172.21.15.12:0/1971759143 wait complete. 2023-07-08T12:38:22.755 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-08T12:38:22.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.880+0000 7ff58d636700 1 Processor -- start 2023-07-08T12:38:22.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.881+0000 7ff58d636700 1 -- start start 2023-07-08T12:38:22.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.881+0000 7ff58d636700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff588156a10 0x7ff58806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.881+0000 7ff58d636700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5880cc240 0x7ff5880cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.881+0000 7ff58d636700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff588160410 con 0x7ff58815d390 2023-07-08T12:38:22.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.881+0000 7ff58d636700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5880ccc00 con 0x7ff588156a10 2023-07-08T12:38:22.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.881+0000 7ff58d636700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff588156de0 con 0x7ff5880cc240 2023-07-08T12:38:22.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.882+0000 7ff5867fc700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff58815d390 0x7ff5881608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36822/0 (socket says 172.21.15.12:36822) 2023-07-08T12:38:22.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.882+0000 7ff5867fc700 1 -- 172.21.15.12:0/3629636034 learned_addr learned my addr 172.21.15.12:0/3629636034 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:22.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.882+0000 7ff586ffd700 1 --2- 172.21.15.12:0/3629636034 >> v2:172.21.15.12:3300/0 conn(0x7ff588156a10 0x7ff58806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.882+0000 7ff5877fe700 1 --2- 172.21.15.12:0/3629636034 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5880cc240 0x7ff5880cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.882+0000 7ff586ffd700 1 -- 172.21.15.12:0/3629636034 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5880cc240 msgr2=0x7ff5880cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.882+0000 7ff586ffd700 1 --2- 172.21.15.12:0/3629636034 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5880cc240 0x7ff5880cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.882+0000 7ff586ffd700 1 -- 172.21.15.12:0/3629636034 >> v1:172.21.15.12:6789/0 conn(0x7ff58815d390 legacy=0x7ff5881608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.882+0000 7ff586ffd700 1 -- 172.21.15.12:0/3629636034 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58806ec70 con 0x7ff588156a10 2023-07-08T12:38:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.882+0000 7ff585ffb700 1 -- 172.21.15.12:0/3629636034 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090980235 0 0) 0x7ff588160410 con 0x7ff58815d390 2023-07-08T12:38:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.882+0000 7ff586ffd700 1 --2- 172.21.15.12:0/3629636034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff588156a10 0x7ff58806e0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff570009820 tx=0x7ff570009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adae334858a4a58 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:22.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.882+0000 7ff585ffb700 1 -- 172.21.15.12:0/3629636034 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570005e30 con 0x7ff588156a10 2023-07-08T12:38:22.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.882+0000 7ff585ffb700 1 -- 172.21.15.12:0/3629636034 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff570006030 con 0x7ff588156a10 2023-07-08T12:38:22.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.882+0000 7ff585ffb700 1 -- 172.21.15.12:0/3629636034 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5700063e0 con 0x7ff588156a10 2023-07-08T12:38:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.883+0000 7ff58d636700 1 -- 172.21.15.12:0/3629636034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff588156a10 msgr2=0x7ff58806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.883+0000 7ff58d636700 1 --2- 172.21.15.12:0/3629636034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff588156a10 0x7ff58806e0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff570009820 tx=0x7ff570009b30 comp rx=0 tx=0).stop 2023-07-08T12:38:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.883+0000 7ff58d636700 1 -- 172.21.15.12:0/3629636034 shutdown_connections 2023-07-08T12:38:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.883+0000 7ff58d636700 1 --2- 172.21.15.12:0/3629636034 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5880cc240 0x7ff5880cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.883+0000 7ff58d636700 1 --2- 172.21.15.12:0/3629636034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff588156a10 0x7ff58806e0c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.883+0000 7ff58d636700 1 -- 172.21.15.12:0/3629636034 >> 172.21.15.12:0/3629636034 conn(0x7ff588098b50 msgr2=0x7ff58806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.883+0000 7ff58d636700 1 -- 172.21.15.12:0/3629636034 shutdown_connections 2023-07-08T12:38:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.883+0000 7ff58d636700 1 -- 172.21.15.12:0/3629636034 wait complete. 2023-07-08T12:38:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.884+0000 7ff58d636700 1 Processor -- start 2023-07-08T12:38:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.884+0000 7ff58d636700 1 -- start start 2023-07-08T12:38:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff58d636700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5880cc240 0x7ff5881f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff58d636700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff588156a10 0x7ff5880b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff58d636700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58815d390 0x7ff58815bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff58d636700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880b4000 con 0x7ff58815d390 2023-07-08T12:38:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff58d636700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880b42e0 con 0x7ff5880cc240 2023-07-08T12:38:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff58d636700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880b45c0 con 0x7ff588156a10 2023-07-08T12:38:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff5877fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58815d390 0x7ff58815bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff5877fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58815d390 0x7ff58815bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49434/0 (socket says 172.21.15.12:49434) 2023-07-08T12:38:22.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff5877fe700 1 -- 172.21.15.12:0/736153248 learned_addr learned my addr 172.21.15.12:0/736153248 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:22.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff586ffd700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5880cc240 0x7ff5881f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff5867fc700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff588156a10 0x7ff5880b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff5877fe700 1 -- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff588156a10 msgr2=0x7ff5880b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff5877fe700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff588156a10 0x7ff5880b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff5877fe700 1 -- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5880cc240 msgr2=0x7ff5881f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:22.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff5877fe700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5880cc240 0x7ff5881f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:22.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff5877fe700 1 -- 172.21.15.12:0/736153248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff570009430 con 0x7ff58815d390 2023-07-08T12:38:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff5867fc700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff588156a10 0x7ff5880b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.885+0000 7ff5877fe700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58815d390 0x7ff58815bf40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff580007e40 tx=0x7ff58000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.886+0000 7ff57f7fe700 1 -- 172.21.15.12:0/736153248 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58000ede0 con 0x7ff58815d390 2023-07-08T12:38:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.886+0000 7ff57f7fe700 1 -- 172.21.15.12:0/736153248 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff580014420 con 0x7ff58815d390 2023-07-08T12:38:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.886+0000 7ff57f7fe700 1 -- 172.21.15.12:0/736153248 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5800147d0 con 0x7ff58815d390 2023-07-08T12:38:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.886+0000 7ff58d636700 1 -- 172.21.15.12:0/736153248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5880b48a0 con 0x7ff58815d390 2023-07-08T12:38:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.886+0000 7ff58d636700 1 -- 172.21.15.12:0/736153248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5881f2140 con 0x7ff58815d390 2023-07-08T12:38:22.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.887+0000 7ff58d636700 1 -- 172.21.15.12:0/736153248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5880b4dc0 con 0x7ff58815d390 2023-07-08T12:38:22.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.887+0000 7ff57f7fe700 1 -- 172.21.15.12:0/736153248 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff580014930 con 0x7ff58815d390 2023-07-08T12:38:22.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.887+0000 7ff57f7fe700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff56806ace0 0x7ff56806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:22.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.888+0000 7ff57f7fe700 1 -- 172.21.15.12:0/736153248 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff580089e80 con 0x7ff58815d390 2023-07-08T12:38:22.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.888+0000 7ff586ffd700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff56806ace0 0x7ff56806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:22.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.888+0000 7ff586ffd700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff56806ace0 0x7ff56806d190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff570013040 tx=0x7ff570008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:22.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:22.892+0000 7ff57f7fe700 1 -- 172.21.15.12:0/736153248 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff5880b4dc0 con 0x7ff58815d390 2023-07-08T12:38:23.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.087+0000 7ff58d636700 1 -- 172.21.15.12:0/736153248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5880b4dc0 con 0x7ff58815d390 2023-07-08T12:38:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.087+0000 7ff57f7fe700 1 -- 172.21.15.12:0/736153248 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5880b4dc0 con 0x7ff58815d390 2023-07-08T12:38:23.090 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-07-08T12:38:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.089+0000 7ff58d636700 1 -- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff56806ace0 msgr2=0x7ff56806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.089+0000 7ff58d636700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff56806ace0 0x7ff56806d190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff570013040 tx=0x7ff570008070 comp rx=0 tx=0).stop 2023-07-08T12:38:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.089+0000 7ff58d636700 1 -- 172.21.15.12:0/736153248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58815d390 msgr2=0x7ff58815bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.089+0000 7ff58d636700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58815d390 0x7ff58815bf40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff580007e40 tx=0x7ff58000e3c0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.089+0000 7ff58d636700 1 -- 172.21.15.12:0/736153248 shutdown_connections 2023-07-08T12:38:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.089+0000 7ff58d636700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff56806ace0 0x7ff56806d190 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.089+0000 7ff58d636700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58815d390 0x7ff58815bf40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.089+0000 7ff58d636700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5880cc240 0x7ff5881f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.089+0000 7ff58d636700 1 --2- 172.21.15.12:0/736153248 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff588156a10 0x7ff5880b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.089+0000 7ff58d636700 1 -- 172.21.15.12:0/736153248 >> 172.21.15.12:0/736153248 conn(0x7ff588098b50 msgr2=0x7ff5880b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:23.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.089+0000 7ff58d636700 1 -- 172.21.15.12:0/736153248 shutdown_connections 2023-07-08T12:38:23.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.089+0000 7ff58d636700 1 -- 172.21.15.12:0/736153248 wait complete. 2023-07-08T12:38:23.103 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-08T12:38:23.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.228+0000 7f118ec40700 1 Processor -- start 2023-07-08T12:38:23.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.228+0000 7f118ec40700 1 -- start start 2023-07-08T12:38:23.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.229+0000 7f118ec40700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f11880aeec0 0x7f118806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:23.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.229+0000 7f118ec40700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f11880cdb80 0x7f11880cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:23.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.229+0000 7f118ec40700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118815d770 con 0x7f118806ebe0 2023-07-08T12:38:23.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.229+0000 7f118ec40700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118815a600 con 0x7f11880aeec0 2023-07-08T12:38:23.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.229+0000 7f118ec40700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118815cb60 con 0x7f11880cdb80 2023-07-08T12:38:23.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.229+0000 7f118c9dc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f11880aeec0 0x7f118806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:23.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.229+0000 7f118c9dc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f11880aeec0 0x7f118806cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49438/0 (socket says 172.21.15.12:49438) 2023-07-08T12:38:23.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.229+0000 7f118c9dc700 1 -- 172.21.15.12:0/1777089699 learned_addr learned my addr 172.21.15.12:0/1777089699 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:23.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.229+0000 7f118d1dd700 1 --2- 172.21.15.12:0/1777089699 >> v2:172.21.15.160:3300/0 conn(0x7f11880cdb80 0x7f11880cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:23.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.229+0000 7f118c9dc700 1 -- 172.21.15.12:0/1777089699 >> v2:172.21.15.160:3300/0 conn(0x7f11880cdb80 msgr2=0x7f11880cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:23.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.229+0000 7f118c9dc700 1 --2- 172.21.15.12:0/1777089699 >> v2:172.21.15.160:3300/0 conn(0x7f11880cdb80 0x7f11880cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.230+0000 7f118c9dc700 1 -- 172.21.15.12:0/1777089699 >> v1:172.21.15.160:6789/0 conn(0x7f118806ebe0 legacy=0x7f11880cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:23.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.230+0000 7f118c9dc700 1 -- 172.21.15.12:0/1777089699 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118815ce40 con 0x7f11880aeec0 2023-07-08T12:38:23.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.230+0000 7f118d1dd700 1 --2- 172.21.15.12:0/1777089699 >> v2:172.21.15.160:3300/0 conn(0x7f11880cdb80 0x7f11880cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:23.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.230+0000 7f118c9dc700 1 --2- 172.21.15.12:0/1777089699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11880aeec0 0x7f118806cff0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f118806fed0 tx=0x7f117c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581b45f96cb24798 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:23.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.230+0000 7f11877fe700 1 -- 172.21.15.12:0/1777089699 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c005e30 con 0x7f11880aeec0 2023-07-08T12:38:23.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.230+0000 7f11877fe700 1 -- 172.21.15.12:0/1777089699 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c004030 con 0x7f11880aeec0 2023-07-08T12:38:23.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.230+0000 7f11877fe700 1 -- 172.21.15.12:0/1777089699 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c0043e0 con 0x7f11880aeec0 2023-07-08T12:38:23.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.230+0000 7f118ec40700 1 -- 172.21.15.12:0/1777089699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11880aeec0 msgr2=0x7f118806cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:23.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.230+0000 7f118ec40700 1 --2- 172.21.15.12:0/1777089699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11880aeec0 0x7f118806cff0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f118806fed0 tx=0x7f117c009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.230+0000 7f118ec40700 1 -- 172.21.15.12:0/1777089699 shutdown_connections 2023-07-08T12:38:23.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.230+0000 7f118ec40700 1 --2- 172.21.15.12:0/1777089699 >> v2:172.21.15.160:3300/0 conn(0x7f11880cdb80 0x7f11880cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.231+0000 7f118ec40700 1 --2- 172.21.15.12:0/1777089699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11880aeec0 0x7f118806cff0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.231+0000 7f118ec40700 1 -- 172.21.15.12:0/1777089699 >> 172.21.15.12:0/1777089699 conn(0x7f1188098b50 msgr2=0x7f118806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:23.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.231+0000 7f118ec40700 1 -- 172.21.15.12:0/1777089699 shutdown_connections 2023-07-08T12:38:23.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.231+0000 7f118ec40700 1 -- 172.21.15.12:0/1777089699 wait complete. 2023-07-08T12:38:23.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.232+0000 7f118ec40700 1 Processor -- start 2023-07-08T12:38:23.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.232+0000 7f118ec40700 1 -- start start 2023-07-08T12:38:23.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.233+0000 7f118ec40700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f118806ebe0 0x7f118814bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:23.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.233+0000 7f118ec40700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11880aeec0 0x7f11880b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:23.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.233+0000 7f118ec40700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f11880cdb80 0x7f11880b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:23.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.233+0000 7f118ec40700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11880b5760 con 0x7f11880aeec0 2023-07-08T12:38:23.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.233+0000 7f118ec40700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118806ad40 con 0x7f118806ebe0 2023-07-08T12:38:23.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.233+0000 7f118ec40700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11880b49a0 con 0x7f11880cdb80 2023-07-08T12:38:23.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.233+0000 7f1187fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11880aeec0 0x7f11880b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:23.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.233+0000 7f1187fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11880aeec0 0x7f11880b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49446/0 (socket says 172.21.15.12:49446) 2023-07-08T12:38:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.233+0000 7f1187fff700 1 -- 172.21.15.12:0/2252147037 learned_addr learned my addr 172.21.15.12:0/2252147037 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f118c9dc700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f118806ebe0 0x7f118814bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f118d1dd700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f11880cdb80 0x7f11880b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f1187fff700 1 -- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f11880cdb80 msgr2=0x7f11880b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f1187fff700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f11880cdb80 0x7f11880b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f1187fff700 1 -- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f118806ebe0 msgr2=0x7f118814bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f1187fff700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f118806ebe0 0x7f118814bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f1187fff700 1 -- 172.21.15.12:0/2252147037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c009500 con 0x7f11880aeec0 2023-07-08T12:38:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f118c9dc700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f118806ebe0 0x7f118814bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f1187fff700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11880aeec0 0x7f11880b1460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f117400c450 tx=0x7f117400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f11857fa700 1 -- 172.21.15.12:0/2252147037 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11740052d0 con 0x7f11880aeec0 2023-07-08T12:38:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f11857fa700 1 -- 172.21.15.12:0/2252147037 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1174005430 con 0x7f11880aeec0 2023-07-08T12:38:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f118ec40700 1 -- 172.21.15.12:0/2252147037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11880b4c80 con 0x7f11880aeec0 2023-07-08T12:38:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f11857fa700 1 -- 172.21.15.12:0/2252147037 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11740038c0 con 0x7f11880aeec0 2023-07-08T12:38:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.234+0000 7f118ec40700 1 -- 172.21.15.12:0/2252147037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11880b5240 con 0x7f11880aeec0 2023-07-08T12:38:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.235+0000 7f118ec40700 1 -- 172.21.15.12:0/2252147037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118804ea60 con 0x7f11880aeec0 2023-07-08T12:38:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.236+0000 7f11857fa700 1 -- 172.21.15.12:0/2252147037 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1174003a20 con 0x7f11880aeec0 2023-07-08T12:38:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.236+0000 7f11857fa700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f116c06ace0 0x7f116c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.236+0000 7f11857fa700 1 -- 172.21.15.12:0/2252147037 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1174088b60 con 0x7f11880aeec0 2023-07-08T12:38:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.236+0000 7f118c9dc700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f116c06ace0 0x7f116c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:23.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.237+0000 7f118c9dc700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f116c06ace0 0x7f116c06d190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f118806fed0 tx=0x7f117c008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:23.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.240+0000 7f11857fa700 1 -- 172.21.15.12:0/2252147037 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f11740538a0 con 0x7f11880aeec0 2023-07-08T12:38:23.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.437+0000 7f118ec40700 1 -- 172.21.15.12:0/2252147037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f118815d0f0 con 0x7f11880aeec0 2023-07-08T12:38:23.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.437+0000 7f11857fa700 1 -- 172.21.15.12:0/2252147037 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1174056ec0 con 0x7f11880aeec0 2023-07-08T12:38:23.440 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:38:23.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.439+0000 7f118ec40700 1 -- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f116c06ace0 msgr2=0x7f116c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:23.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.439+0000 7f118ec40700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f116c06ace0 0x7f116c06d190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f118806fed0 tx=0x7f117c008070 comp rx=0 tx=0).stop 2023-07-08T12:38:23.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.439+0000 7f118ec40700 1 -- 172.21.15.12:0/2252147037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11880aeec0 msgr2=0x7f11880b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:23.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.439+0000 7f118ec40700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11880aeec0 0x7f11880b1460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f117400c450 tx=0x7f117400c810 comp rx=0 tx=0).stop 2023-07-08T12:38:23.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.439+0000 7f118ec40700 1 -- 172.21.15.12:0/2252147037 shutdown_connections 2023-07-08T12:38:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.439+0000 7f118ec40700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f116c06ace0 0x7f116c06d190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.439+0000 7f118ec40700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f11880cdb80 0x7f11880b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.439+0000 7f118ec40700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f118806ebe0 0x7f118814bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.439+0000 7f118ec40700 1 --2- 172.21.15.12:0/2252147037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11880aeec0 0x7f11880b1460 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.439+0000 7f118ec40700 1 -- 172.21.15.12:0/2252147037 >> 172.21.15.12:0/2252147037 conn(0x7f1188098b50 msgr2=0x7f11880ac420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.439+0000 7f118ec40700 1 -- 172.21.15.12:0/2252147037 shutdown_connections 2023-07-08T12:38:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.439+0000 7f118ec40700 1 -- 172.21.15.12:0/2252147037 wait complete. 2023-07-08T12:38:23.454 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-08T12:38:23.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.577+0000 7f68be29e700 1 Processor -- start 2023-07-08T12:38:23.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.577+0000 7f68be29e700 1 -- start start 2023-07-08T12:38:23.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.578+0000 7f68be29e700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f68b8159c30 0x7f68b80cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:23.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.578+0000 7f68be29e700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f68b8160790 0x7f68b806ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:23.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.578+0000 7f68be29e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68b8069f30 0x7f68b806a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:23.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.578+0000 7f68be29e700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b815c730 con 0x7f68b8159c30 2023-07-08T12:38:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.578+0000 7f68be29e700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b80cf590 con 0x7f68b8160790 2023-07-08T12:38:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.578+0000 7f68be29e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b806a8e0 con 0x7f68b8069f30 2023-07-08T12:38:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.578+0000 7f68b7fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f68b8159c30 0x7f68b80cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.578+0000 7f68b7fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f68b8159c30 0x7f68b80cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49452/0 (socket says 172.21.15.12:49452) 2023-07-08T12:38:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.578+0000 7f68b7fff700 1 -- 172.21.15.12:0/1238473947 learned_addr learned my addr 172.21.15.12:0/1238473947 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.578+0000 7f68b77fe700 1 --2- 172.21.15.12:0/1238473947 >> v2:172.21.15.160:3300/0 conn(0x7f68b8160790 0x7f68b806ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.579+0000 7f68bc83b700 1 --2- 172.21.15.12:0/1238473947 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68b8069f30 0x7f68b806a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.579+0000 7f68b7fff700 1 -- 172.21.15.12:0/1238473947 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68b8069f30 msgr2=0x7f68b806a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.579+0000 7f68b7fff700 1 --2- 172.21.15.12:0/1238473947 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68b8069f30 0x7f68b806a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.579+0000 7f68b7fff700 1 -- 172.21.15.12:0/1238473947 >> v2:172.21.15.160:3300/0 conn(0x7f68b8160790 msgr2=0x7f68b806ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.579+0000 7f68b7fff700 1 --2- 172.21.15.12:0/1238473947 >> v2:172.21.15.160:3300/0 conn(0x7f68b8160790 0x7f68b806ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.579+0000 7f68b7fff700 1 -- 172.21.15.12:0/1238473947 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b815c890 con 0x7f68b8159c30 2023-07-08T12:38:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.579+0000 7f68b7fff700 1 --2- 172.21.15.12:0/1238473947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b8159c30 0x7f68b80cf050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f68b80b2370 tx=0x7f68ac009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c2314b075f0f82 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.579+0000 7f68b6ffd700 1 -- 172.21.15.12:0/1238473947 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00b9e0 con 0x7f68b8159c30 2023-07-08T12:38:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.579+0000 7f68b6ffd700 1 -- 172.21.15.12:0/1238473947 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ac00bb40 con 0x7f68b8159c30 2023-07-08T12:38:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.579+0000 7f68b6ffd700 1 -- 172.21.15.12:0/1238473947 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac009e70 con 0x7f68b8159c30 2023-07-08T12:38:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.579+0000 7f68be29e700 1 -- 172.21.15.12:0/1238473947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b8159c30 msgr2=0x7f68b80cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.579+0000 7f68be29e700 1 --2- 172.21.15.12:0/1238473947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b8159c30 0x7f68b80cf050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f68b80b2370 tx=0x7f68ac009c30 comp rx=0 tx=0).stop 2023-07-08T12:38:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.580+0000 7f68be29e700 1 -- 172.21.15.12:0/1238473947 shutdown_connections 2023-07-08T12:38:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.580+0000 7f68be29e700 1 --2- 172.21.15.12:0/1238473947 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68b8069f30 0x7f68b806a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.580+0000 7f68be29e700 1 --2- 172.21.15.12:0/1238473947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b8159c30 0x7f68b80cf050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.580+0000 7f68be29e700 1 --2- 172.21.15.12:0/1238473947 >> v2:172.21.15.160:3300/0 conn(0x7f68b8160790 0x7f68b806ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.580+0000 7f68be29e700 1 -- 172.21.15.12:0/1238473947 >> 172.21.15.12:0/1238473947 conn(0x7f68b80cc290 msgr2=0x7f68b80cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.580+0000 7f68be29e700 1 -- 172.21.15.12:0/1238473947 shutdown_connections 2023-07-08T12:38:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.580+0000 7f68be29e700 1 -- 172.21.15.12:0/1238473947 wait complete. 2023-07-08T12:38:23.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.581+0000 7f68be29e700 1 Processor -- start 2023-07-08T12:38:23.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.581+0000 7f68be29e700 1 -- start start 2023-07-08T12:38:23.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68be29e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b8069f30 0x7f68b80b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:23.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68be29e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68b8159c30 0x7f68b80b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:23.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68be29e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68b8160790 0x7f68b81e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:23.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68be29e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b814b510 con 0x7f68b8069f30 2023-07-08T12:38:23.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68be29e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b80b0aa0 con 0x7f68b8160790 2023-07-08T12:38:23.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68be29e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b814df60 con 0x7f68b8159c30 2023-07-08T12:38:23.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68b7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b8069f30 0x7f68b80b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:23.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68b7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b8069f30 0x7f68b80b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49458/0 (socket says 172.21.15.12:49458) 2023-07-08T12:38:23.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68b7fff700 1 -- 172.21.15.12:0/471281552 learned_addr learned my addr 172.21.15.12:0/471281552 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:23.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68bc83b700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68b8160790 0x7f68b81e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:23.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68b77fe700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68b8159c30 0x7f68b80b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:23.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68b7fff700 1 -- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68b8159c30 msgr2=0x7f68b80b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:23.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68b7fff700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68b8159c30 0x7f68b80b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68b7fff700 1 -- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68b8160790 msgr2=0x7f68b81e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:23.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68b7fff700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68b8160790 0x7f68b81e7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68b7fff700 1 -- 172.21.15.12:0/471281552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac009550 con 0x7f68b8069f30 2023-07-08T12:38:23.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68bc83b700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68b8160790 0x7f68b81e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:23.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68b7fff700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b8069f30 0x7f68b80b0560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f68ac000c00 tx=0x7f68ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:23.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68b4ff9700 1 -- 172.21.15.12:0/471281552 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac004400 con 0x7f68b8069f30 2023-07-08T12:38:23.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.582+0000 7f68b4ff9700 1 -- 172.21.15.12:0/471281552 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ac01f070 con 0x7f68b8069f30 2023-07-08T12:38:23.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.583+0000 7f68be29e700 1 -- 172.21.15.12:0/471281552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b80b1850 con 0x7f68b8069f30 2023-07-08T12:38:23.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.583+0000 7f68b4ff9700 1 -- 172.21.15.12:0/471281552 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0038e0 con 0x7f68b8069f30 2023-07-08T12:38:23.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.583+0000 7f68be29e700 1 -- 172.21.15.12:0/471281552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b80af270 con 0x7f68b8069f30 2023-07-08T12:38:23.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.584+0000 7f68be29e700 1 -- 172.21.15.12:0/471281552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b80b03b0 con 0x7f68b8069f30 2023-07-08T12:38:23.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.584+0000 7f68b4ff9700 1 -- 172.21.15.12:0/471281552 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f68ac021030 con 0x7f68b8069f30 2023-07-08T12:38:23.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.584+0000 7f68b4ff9700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f68a406ace0 0x7f68a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:23.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.584+0000 7f68b4ff9700 1 -- 172.21.15.12:0/471281552 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f68ac08a8f0 con 0x7f68b8069f30 2023-07-08T12:38:23.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.585+0000 7f68b77fe700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f68a406ace0 0x7f68a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:23.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.585+0000 7f68b77fe700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f68a406ace0 0x7f68a406d190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f68b814e110 tx=0x7f68a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:23.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.589+0000 7f68b4ff9700 1 -- 172.21.15.12:0/471281552 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f68ac055740 con 0x7f68b8069f30 2023-07-08T12:38:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.800+0000 7f68be29e700 1 -- 172.21.15.12:0/471281552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f68b80b1850 con 0x7f68b8069f30 2023-07-08T12:38:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.800+0000 7f68b4ff9700 1 -- 172.21.15.12:0/471281552 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f68b80b1850 con 0x7f68b8069f30 2023-07-08T12:38:23.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.803+0000 7f68be29e700 1 -- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f68a406ace0 msgr2=0x7f68a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:23.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.803+0000 7f68be29e700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f68a406ace0 0x7f68a406d190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f68b814e110 tx=0x7f68a8008040 comp rx=0 tx=0).stop 2023-07-08T12:38:23.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.803+0000 7f68be29e700 1 -- 172.21.15.12:0/471281552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b8069f30 msgr2=0x7f68b80b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:23.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.803+0000 7f68be29e700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b8069f30 0x7f68b80b0560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f68ac000c00 tx=0x7f68ac004750 comp rx=0 tx=0).stop 2023-07-08T12:38:23.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.803+0000 7f68be29e700 1 -- 172.21.15.12:0/471281552 shutdown_connections 2023-07-08T12:38:23.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.803+0000 7f68be29e700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f68a406ace0 0x7f68a406d190 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.803+0000 7f68be29e700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68b8160790 0x7f68b81e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.803+0000 7f68be29e700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68b8069f30 0x7f68b80b0560 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.803+0000 7f68be29e700 1 --2- 172.21.15.12:0/471281552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68b8159c30 0x7f68b80b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:23.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.803+0000 7f68be29e700 1 -- 172.21.15.12:0/471281552 >> 172.21.15.12:0/471281552 conn(0x7f68b80cc290 msgr2=0x7f68b815a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:23.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.803+0000 7f68be29e700 1 -- 172.21.15.12:0/471281552 shutdown_connections 2023-07-08T12:38:23.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:23.803+0000 7f68be29e700 1 -- 172.21.15.12:0/471281552 wait complete. 2023-07-08T12:38:23.807 INFO:teuthology.orchestra.run.smithi012.stderr:listed 0 entries 2023-07-08T12:38:23.817 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi012.front.sepia.ceph.com... 2023-07-08T12:38:23.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:23.819 DEBUG:teuthology.orchestra.run.smithi012:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-07-08T12:38:23.854 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[138846]: fuse finished with error 0 and tester_r 0 2023-07-08T12:38:23.856 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T12:38:29.958 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2023-07-08T12:38:29.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:29.960 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-07-08T12:38:29.993 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:30.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.170+0000 7f230977d700 1 Processor -- start 2023-07-08T12:38:30.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.170+0000 7f230977d700 1 -- start start 2023-07-08T12:38:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.170+0000 7f230977d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2304159c30 0x7f23040cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.170+0000 7f230977d700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230415c890 con 0x7f2304160790 2023-07-08T12:38:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.170+0000 7f230977d700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230406f620 con 0x7f2304069f30 2023-07-08T12:38:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.170+0000 7f230977d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230415c730 con 0x7f2304159c30 2023-07-08T12:38:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f23027fc700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2304160790 0x7f230406ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36864/0 (socket says 172.21.15.12:36864) 2023-07-08T12:38:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f23027fc700 1 -- 172.21.15.12:0/2892599196 learned_addr learned my addr 172.21.15.12:0/2892599196 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2302ffd700 1 --2- 172.21.15.12:0/2892599196 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2304159c30 0x7f23040cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487795571 0 0) 0x7f230415c890 con 0x7f2304160790 2023-07-08T12:38:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22ec0035b0 con 0x7f2304160790 2023-07-08T12:38:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297217911 0 0) 0x7f230406f620 con 0x7f2304069f30 2023-07-08T12:38:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f230415c890 con 0x7f2304069f30 2023-07-08T12:38:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3967851338 0 0) 0x7f22ec0035b0 con 0x7f2304160790 2023-07-08T12:38:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2304159c30 msgr2=0x7f23040cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 --2- 172.21.15.12:0/2892599196 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2304159c30 0x7f23040cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 >> v1:172.21.15.160:6789/0 conn(0x7f2304069f30 legacy=0x7f230415cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2304160c90 con 0x7f2304160790 2023-07-08T12:38:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f22f8001670 con 0x7f2304160790 2023-07-08T12:38:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 >> v1:172.21.15.12:6789/0 conn(0x7f2304160790 legacy=0x7f230406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 --2- 172.21.15.12:0/2892599196 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ec003bc0 0x7f22ec003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 --2- 172.21.15.12:0/2892599196 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2304159c30 0x7f22ec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 --2- 172.21.15.12:0/2892599196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec004d90 0x7f22ec005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230415c730 con 0x7f22ec003bc0 2023-07-08T12:38:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec00bac0 con 0x7f22ec004d90 2023-07-08T12:38:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec00bc30 con 0x7f2304159c30 2023-07-08T12:38:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f2302ffd700 1 --2- 172.21.15.12:0/2892599196 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ec003bc0 0x7f22ec003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f23037fe700 1 --2- 172.21.15.12:0/2892599196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec004d90 0x7f22ec005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.171+0000 7f23027fc700 1 --2- 172.21.15.12:0/2892599196 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2304159c30 0x7f22ec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.172+0000 7f2302ffd700 1 -- 172.21.15.12:0/2892599196 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2304159c30 msgr2=0x7f22ec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.172+0000 7f2302ffd700 1 --2- 172.21.15.12:0/2892599196 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2304159c30 0x7f22ec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.172+0000 7f2302ffd700 1 -- 172.21.15.12:0/2892599196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec004d90 msgr2=0x7f22ec005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.172+0000 7f2302ffd700 1 --2- 172.21.15.12:0/2892599196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec004d90 0x7f22ec005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.172+0000 7f2302ffd700 1 -- 172.21.15.12:0/2892599196 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec0035b0 con 0x7f22ec003bc0 2023-07-08T12:38:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.172+0000 7f2302ffd700 1 --2- 172.21.15.12:0/2892599196 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ec003bc0 0x7f22ec003fb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009b60 tx=0x7f22ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e287ce7cfc4525 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.172+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f400e660 con 0x7f22ec003bc0 2023-07-08T12:38:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.172+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f400e7c0 con 0x7f22ec003bc0 2023-07-08T12:38:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.172+0000 7f2301ffb700 1 -- 172.21.15.12:0/2892599196 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f400eb30 con 0x7f22ec003bc0 2023-07-08T12:38:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.173+0000 7f230977d700 1 -- 172.21.15.12:0/2892599196 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ec003bc0 msgr2=0x7f22ec003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.173+0000 7f230977d700 1 --2- 172.21.15.12:0/2892599196 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ec003bc0 0x7f22ec003fb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009b60 tx=0x7f22ec005880 comp rx=0 tx=0).stop 2023-07-08T12:38:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.173+0000 7f23037fe700 1 -- 172.21.15.12:0/2892599196 reap_dead start 2023-07-08T12:38:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.173+0000 7f230977d700 1 -- 172.21.15.12:0/2892599196 shutdown_connections 2023-07-08T12:38:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.173+0000 7f230977d700 1 -- 172.21.15.12:0/2892599196 >> 172.21.15.12:0/2892599196 conn(0x7f23040cc290 msgr2=0x7f23040cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.173+0000 7f230977d700 1 -- 172.21.15.12:0/2892599196 shutdown_connections 2023-07-08T12:38:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.173+0000 7f230977d700 1 -- 172.21.15.12:0/2892599196 wait complete. 2023-07-08T12:38:30.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.175+0000 7f230977d700 1 Processor -- start 2023-07-08T12:38:30.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.175+0000 7f230977d700 1 -- start start 2023-07-08T12:38:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.175+0000 7f230977d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec003bc0 0x7f2304154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.175+0000 7f230977d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23040b09a0 0x7f23040af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.175+0000 7f230977d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23041541b0 0x7f23040b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.175+0000 7f230977d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23040af900 con 0x7f23041541b0 2023-07-08T12:38:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.175+0000 7f230977d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23040afbe0 con 0x7f22ec003bc0 2023-07-08T12:38:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.175+0000 7f230977d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23040afec0 con 0x7f23040b09a0 2023-07-08T12:38:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.175+0000 7f23037fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23041541b0 0x7f23040b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.175+0000 7f23037fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23041541b0 0x7f23040b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49482/0 (socket says 172.21.15.12:49482) 2023-07-08T12:38:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.175+0000 7f23037fe700 1 -- 172.21.15.12:0/826757397 learned_addr learned my addr 172.21.15.12:0/826757397 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.176+0000 7f2302ffd700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec003bc0 0x7f2304154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.176+0000 7f23027fc700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23040b09a0 0x7f23040af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.176+0000 7f23037fe700 1 -- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23040b09a0 msgr2=0x7f23040af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.176+0000 7f23037fe700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23040b09a0 0x7f23040af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.176+0000 7f23037fe700 1 -- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec003bc0 msgr2=0x7f2304154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.176+0000 7f23037fe700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec003bc0 0x7f2304154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.176+0000 7f23037fe700 1 -- 172.21.15.12:0/826757397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23040b01a0 con 0x7f23041541b0 2023-07-08T12:38:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.176+0000 7f23037fe700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23041541b0 0x7f23040b1820 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f23040b2370 tx=0x7f22fc00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.176+0000 7f22eb7fe700 1 -- 172.21.15.12:0/826757397 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc003c10 con 0x7f23041541b0 2023-07-08T12:38:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.176+0000 7f22eb7fe700 1 -- 172.21.15.12:0/826757397 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22fc003d70 con 0x7f23041541b0 2023-07-08T12:38:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.176+0000 7f230977d700 1 -- 172.21.15.12:0/826757397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23040b0480 con 0x7f23041541b0 2023-07-08T12:38:30.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.176+0000 7f22eb7fe700 1 -- 172.21.15.12:0/826757397 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc00dbd0 con 0x7f23041541b0 2023-07-08T12:38:30.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.176+0000 7f230977d700 1 -- 172.21.15.12:0/826757397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2304155090 con 0x7f23041541b0 2023-07-08T12:38:30.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.177+0000 7f230977d700 1 -- 172.21.15.12:0/826757397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230404ea60 con 0x7f23041541b0 2023-07-08T12:38:30.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.178+0000 7f22eb7fe700 1 -- 172.21.15.12:0/826757397 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f22fc00dd30 con 0x7f23041541b0 2023-07-08T12:38:30.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.178+0000 7f22eb7fe700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f22f006ace0 0x7f22f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.178+0000 7f22eb7fe700 1 -- 172.21.15.12:0/826757397 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f22fc0899f0 con 0x7f23041541b0 2023-07-08T12:38:30.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.178+0000 7f2302ffd700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f22f006ace0 0x7f22f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.179+0000 7f2302ffd700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f22f006ace0 0x7f22f006d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23040c6790 tx=0x7f22f4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:30.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.182+0000 7f22eb7fe700 1 -- 172.21.15.12:0/826757397 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f22fc054840 con 0x7f23041541b0 2023-07-08T12:38:30.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.381+0000 7f230977d700 1 -- 172.21.15.12:0/826757397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23040b5790 con 0x7f23041541b0 2023-07-08T12:38:30.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.382+0000 7f22eb7fe700 1 -- 172.21.15.12:0/826757397 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f22fc057e60 con 0x7f23041541b0 2023-07-08T12:38:30.384 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:30.384 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:12.993803+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.385+0000 7f230977d700 1 -- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f22f006ace0 msgr2=0x7f22f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.385+0000 7f230977d700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f22f006ace0 0x7f22f006d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23040c6790 tx=0x7f22f4008030 comp rx=0 tx=0).stop 2023-07-08T12:38:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.385+0000 7f230977d700 1 -- 172.21.15.12:0/826757397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23041541b0 msgr2=0x7f23040b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.385+0000 7f230977d700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23041541b0 0x7f23040b1820 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f23040b2370 tx=0x7f22fc00d430 comp rx=0 tx=0).stop 2023-07-08T12:38:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.385+0000 7f230977d700 1 -- 172.21.15.12:0/826757397 shutdown_connections 2023-07-08T12:38:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.385+0000 7f230977d700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f22f006ace0 0x7f22f006d190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.385+0000 7f230977d700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23041541b0 0x7f23040b1820 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.385+0000 7f230977d700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec003bc0 0x7f2304154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.385+0000 7f230977d700 1 --2- 172.21.15.12:0/826757397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23040b09a0 0x7f23040af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.385+0000 7f230977d700 1 -- 172.21.15.12:0/826757397 >> 172.21.15.12:0/826757397 conn(0x7f23040cc290 msgr2=0x7f230415b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.385+0000 7f230977d700 1 -- 172.21.15.12:0/826757397 shutdown_connections 2023-07-08T12:38:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.385+0000 7f230977d700 1 -- 172.21.15.12:0/826757397 wait complete. 2023-07-08T12:38:30.398 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:30.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.522+0000 7f658e87a700 1 Processor -- start 2023-07-08T12:38:30.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.522+0000 7f658e87a700 1 -- start start 2023-07-08T12:38:30.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f658e87a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6588070bb0 0x7f658815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f658e87a700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f658806b640 0x7f65880c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f658e87a700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658806b1b0 con 0x7f65880cd530 2023-07-08T12:38:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f658e87a700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f658815d1f0 con 0x7f6588070bb0 2023-07-08T12:38:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f658e87a700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f658806a5a0 con 0x7f658806b640 2023-07-08T12:38:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f6587fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6588070bb0 0x7f658815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f6587fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6588070bb0 0x7f658815ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49486/0 (socket says 172.21.15.12:49486) 2023-07-08T12:38:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f6587fff700 1 -- 172.21.15.12:0/1252104060 learned_addr learned my addr 172.21.15.12:0/1252104060 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:30.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f6587fff700 1 -- 172.21.15.12:0/1252104060 >> v2:172.21.15.160:3300/0 conn(0x7f658806b640 msgr2=0x7f65880c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:30.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f658ce17700 1 --2- 172.21.15.12:0/1252104060 >> v2:172.21.15.160:3300/0 conn(0x7f658806b640 0x7f65880c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f6587fff700 1 --2- 172.21.15.12:0/1252104060 >> v2:172.21.15.160:3300/0 conn(0x7f658806b640 0x7f65880c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f6587fff700 1 -- 172.21.15.12:0/1252104060 >> v1:172.21.15.160:6789/0 conn(0x7f65880cd530 legacy=0x7f6588070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:30.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f6587fff700 1 -- 172.21.15.12:0/1252104060 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658806a880 con 0x7f6588070bb0 2023-07-08T12:38:30.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f6587fff700 1 --2- 172.21.15.12:0/1252104060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6588070bb0 0x7f658815ccb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f658806e9c0 tx=0x7f657800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3a5ee707731629 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f6586ffd700 1 -- 172.21.15.12:0/1252104060 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657800dbe0 con 0x7f6588070bb0 2023-07-08T12:38:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f6586ffd700 1 -- 172.21.15.12:0/1252104060 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657800dd40 con 0x7f6588070bb0 2023-07-08T12:38:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.523+0000 7f6586ffd700 1 -- 172.21.15.12:0/1252104060 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6578004270 con 0x7f6588070bb0 2023-07-08T12:38:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.524+0000 7f658e87a700 1 -- 172.21.15.12:0/1252104060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6588070bb0 msgr2=0x7f658815ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.524+0000 7f658e87a700 1 --2- 172.21.15.12:0/1252104060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6588070bb0 0x7f658815ccb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f658806e9c0 tx=0x7f657800be70 comp rx=0 tx=0).stop 2023-07-08T12:38:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.524+0000 7f658e87a700 1 -- 172.21.15.12:0/1252104060 shutdown_connections 2023-07-08T12:38:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.524+0000 7f658e87a700 1 --2- 172.21.15.12:0/1252104060 >> v2:172.21.15.160:3300/0 conn(0x7f658806b640 0x7f65880c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.524+0000 7f658e87a700 1 --2- 172.21.15.12:0/1252104060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6588070bb0 0x7f658815ccb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.524+0000 7f658e87a700 1 -- 172.21.15.12:0/1252104060 >> 172.21.15.12:0/1252104060 conn(0x7f65880cb9f0 msgr2=0x7f65880cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.524+0000 7f658e87a700 1 -- 172.21.15.12:0/1252104060 shutdown_connections 2023-07-08T12:38:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.524+0000 7f658e87a700 1 -- 172.21.15.12:0/1252104060 wait complete. 2023-07-08T12:38:30.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.525+0000 7f658e87a700 1 Processor -- start 2023-07-08T12:38:30.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.525+0000 7f658e87a700 1 -- start start 2023-07-08T12:38:30.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.526+0000 7f658e87a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f658806b640 0x7f65880ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.526+0000 7f658e87a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6588070bb0 0x7f6588158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.526+0000 7f658e87a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65880cd530 0x7f65880b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.526+0000 7f658e87a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65880abe90 con 0x7f658806b640 2023-07-08T12:38:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.526+0000 7f658e87a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65880ac170 con 0x7f65880cd530 2023-07-08T12:38:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.526+0000 7f658e87a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65880ac450 con 0x7f6588070bb0 2023-07-08T12:38:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.526+0000 7f6587fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f658806b640 0x7f65880ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.526+0000 7f6587fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f658806b640 0x7f65880ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49492/0 (socket says 172.21.15.12:49492) 2023-07-08T12:38:30.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.526+0000 7f6587fff700 1 -- 172.21.15.12:0/1836399226 learned_addr learned my addr 172.21.15.12:0/1836399226 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:30.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.526+0000 7f658ce17700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65880cd530 0x7f65880b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.526+0000 7f65877fe700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6588070bb0 0x7f6588158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.526+0000 7f6587fff700 1 -- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6588070bb0 msgr2=0x7f6588158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.527+0000 7f6587fff700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6588070bb0 0x7f6588158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.527+0000 7f6587fff700 1 -- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65880cd530 msgr2=0x7f65880b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.527+0000 7f6587fff700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65880cd530 0x7f65880b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.527+0000 7f6587fff700 1 -- 172.21.15.12:0/1836399226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657800b7f0 con 0x7f658806b640 2023-07-08T12:38:30.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.527+0000 7f658ce17700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65880cd530 0x7f65880b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:30.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.527+0000 7f6587fff700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f658806b640 0x7f65880ab8b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f657800beb0 tx=0x7f6578004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:30.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.527+0000 7f6584ff9700 1 -- 172.21.15.12:0/1836399226 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6578009040 con 0x7f658806b640 2023-07-08T12:38:30.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.527+0000 7f6584ff9700 1 -- 172.21.15.12:0/1836399226 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6578003bd0 con 0x7f658806b640 2023-07-08T12:38:30.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.527+0000 7f6584ff9700 1 -- 172.21.15.12:0/1836399226 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6578021420 con 0x7f658806b640 2023-07-08T12:38:30.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.527+0000 7f658e87a700 1 -- 172.21.15.12:0/1836399226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65880acb30 con 0x7f658806b640 2023-07-08T12:38:30.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.527+0000 7f658e87a700 1 -- 172.21.15.12:0/1836399226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6588159470 con 0x7f658806b640 2023-07-08T12:38:30.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.528+0000 7f658e87a700 1 -- 172.21.15.12:0/1836399226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658804ea60 con 0x7f658806b640 2023-07-08T12:38:30.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.528+0000 7f6584ff9700 1 -- 172.21.15.12:0/1836399226 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6578015070 con 0x7f658806b640 2023-07-08T12:38:30.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.529+0000 7f6584ff9700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f657406ace0 0x7f657406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.529+0000 7f6584ff9700 1 -- 172.21.15.12:0/1836399226 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f657808b4d0 con 0x7f658806b640 2023-07-08T12:38:30.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.529+0000 7f65877fe700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f657406ace0 0x7f657406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.529+0000 7f65877fe700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f657406ace0 0x7f657406d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f65880b0010 tx=0x7f657c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:30.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.533+0000 7f6584ff9700 1 -- 172.21.15.12:0/1836399226 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6578059e40 con 0x7f658806b640 2023-07-08T12:38:30.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.733+0000 7f658e87a700 1 -- 172.21.15.12:0/1836399226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f658804ea60 con 0x7f658806b640 2023-07-08T12:38:30.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.734+0000 7f6584ff9700 1 -- 172.21.15.12:0/1836399226 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f65780599d0 con 0x7f658806b640 2023-07-08T12:38:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:30.736 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:12.993803+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.736+0000 7f658e87a700 1 -- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f657406ace0 msgr2=0x7f657406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.736+0000 7f658e87a700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f657406ace0 0x7f657406d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f65880b0010 tx=0x7f657c009040 comp rx=0 tx=0).stop 2023-07-08T12:38:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.736+0000 7f658e87a700 1 -- 172.21.15.12:0/1836399226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f658806b640 msgr2=0x7f65880ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.736+0000 7f658e87a700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f658806b640 0x7f65880ab8b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f657800beb0 tx=0x7f6578004750 comp rx=0 tx=0).stop 2023-07-08T12:38:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.736+0000 7f658e87a700 1 -- 172.21.15.12:0/1836399226 shutdown_connections 2023-07-08T12:38:30.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.736+0000 7f658e87a700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f657406ace0 0x7f657406d190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.736+0000 7f658e87a700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65880cd530 0x7f65880b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.736+0000 7f658e87a700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f658806b640 0x7f65880ab8b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.736+0000 7f658e87a700 1 --2- 172.21.15.12:0/1836399226 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6588070bb0 0x7f6588158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.736+0000 7f658e87a700 1 -- 172.21.15.12:0/1836399226 >> 172.21.15.12:0/1836399226 conn(0x7f65880cb9f0 msgr2=0x7f65880cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:30.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.736+0000 7f658e87a700 1 -- 172.21.15.12:0/1836399226 shutdown_connections 2023-07-08T12:38:30.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.736+0000 7f658e87a700 1 -- 172.21.15.12:0/1836399226 wait complete. 2023-07-08T12:38:30.751 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-08T12:38:30.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.878+0000 7f6fac9a2700 1 Processor -- start 2023-07-08T12:38:30.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.878+0000 7f6fac9a2700 1 -- start start 2023-07-08T12:38:30.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.879+0000 7f6fac9a2700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6fa81562b0 0x7f6fa8156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.879+0000 7f6fac9a2700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa8156bc0 con 0x7f6fa8069f30 2023-07-08T12:38:30.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.879+0000 7f6fac9a2700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa80cceb0 con 0x7f6fa80ceca0 2023-07-08T12:38:30.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.879+0000 7f6fac9a2700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa80cf430 con 0x7f6fa81562b0 2023-07-08T12:38:30.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.879+0000 7f6fa77fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6fa81562b0 0x7f6fa8156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.879+0000 7f6fa7fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6fa8069f30 0x7f6fa80cacf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36900/0 (socket says 172.21.15.12:36900) 2023-07-08T12:38:30.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.879+0000 7f6fa7fff700 1 -- 172.21.15.12:0/3766139635 learned_addr learned my addr 172.21.15.12:0/3766139635 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:30.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.879+0000 7f6fa77fe700 1 -- 172.21.15.12:0/3766139635 >> v1:172.21.15.160:6789/0 conn(0x7f6fa80ceca0 legacy=0x7f6fa815cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:30.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.879+0000 7f6fa77fe700 1 -- 172.21.15.12:0/3766139635 >> v1:172.21.15.12:6789/0 conn(0x7f6fa8069f30 legacy=0x7f6fa80cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.879+0000 7f6fa77fe700 1 -- 172.21.15.12:0/3766139635 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa80b1240 con 0x7f6fa81562b0 2023-07-08T12:38:30.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.880+0000 7f6fa77fe700 1 --2- 172.21.15.12:0/3766139635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa81562b0 0x7f6fa8156680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009840 tx=0x7f6f98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e41674d56dbc8c server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:30.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.880+0000 7f6fa67fc700 1 -- 172.21.15.12:0/3766139635 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98004230 con 0x7f6fa81562b0 2023-07-08T12:38:30.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.880+0000 7f6fa67fc700 1 -- 172.21.15.12:0/3766139635 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f98004390 con 0x7f6fa81562b0 2023-07-08T12:38:30.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.880+0000 7f6fa67fc700 1 -- 172.21.15.12:0/3766139635 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f980036a0 con 0x7f6fa81562b0 2023-07-08T12:38:30.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.880+0000 7f6fac9a2700 1 -- 172.21.15.12:0/3766139635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa81562b0 msgr2=0x7f6fa8156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.880+0000 7f6fac9a2700 1 --2- 172.21.15.12:0/3766139635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa81562b0 0x7f6fa8156680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009840 tx=0x7f6f98009b50 comp rx=0 tx=0).stop 2023-07-08T12:38:30.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.880+0000 7f6fac9a2700 1 -- 172.21.15.12:0/3766139635 shutdown_connections 2023-07-08T12:38:30.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.880+0000 7f6fac9a2700 1 --2- 172.21.15.12:0/3766139635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa81562b0 0x7f6fa8156680 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.880+0000 7f6fac9a2700 1 -- 172.21.15.12:0/3766139635 >> 172.21.15.12:0/3766139635 conn(0x7f6fa806bcd0 msgr2=0x7f6fa806d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:30.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.880+0000 7f6fac9a2700 1 -- 172.21.15.12:0/3766139635 shutdown_connections 2023-07-08T12:38:30.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.880+0000 7f6fac9a2700 1 -- 172.21.15.12:0/3766139635 wait complete. 2023-07-08T12:38:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.881+0000 7f6fac9a2700 1 Processor -- start 2023-07-08T12:38:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.882+0000 7f6fac9a2700 1 -- start start 2023-07-08T12:38:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.882+0000 7f6fac9a2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa8069f30 0x7f6fa814fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.882+0000 7f6fac9a2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fa80ceca0 0x7f6fa814f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.882+0000 7f6fac9a2700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fa81562b0 0x7f6fa80c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.882+0000 7f6fac9a2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa814f940 con 0x7f6fa8069f30 2023-07-08T12:38:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.882+0000 7f6fac9a2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa81e3f50 con 0x7f6fa80ceca0 2023-07-08T12:38:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.882+0000 7f6fac9a2700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa80c4070 con 0x7f6fa81562b0 2023-07-08T12:38:30.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.882+0000 7f6fa77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa8069f30 0x7f6fa814fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.882+0000 7f6fa77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa8069f30 0x7f6fa814fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49506/0 (socket says 172.21.15.12:49506) 2023-07-08T12:38:30.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.882+0000 7f6fa77fe700 1 -- 172.21.15.12:0/2723859619 learned_addr learned my addr 172.21.15.12:0/2723859619 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:30.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.883+0000 7f6fa6ffd700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fa80ceca0 0x7f6fa814f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.883+0000 7f6fa7fff700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fa81562b0 0x7f6fa80c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.883+0000 7f6fa77fe700 1 -- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fa81562b0 msgr2=0x7f6fa80c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.883+0000 7f6fa77fe700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fa81562b0 0x7f6fa80c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.883+0000 7f6fa77fe700 1 -- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fa80ceca0 msgr2=0x7f6fa814f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.883+0000 7f6fa77fe700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fa80ceca0 0x7f6fa814f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.883+0000 7f6fa77fe700 1 -- 172.21.15.12:0/2723859619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f980094a0 con 0x7f6fa8069f30 2023-07-08T12:38:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.883+0000 7f6fa77fe700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa8069f30 0x7f6fa814fab0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6f980090b0 tx=0x7f6f980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.883+0000 7f6f9ffff700 1 -- 172.21.15.12:0/2723859619 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98003ea0 con 0x7f6fa8069f30 2023-07-08T12:38:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.883+0000 7f6f9ffff700 1 -- 172.21.15.12:0/2723859619 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f98011630 con 0x7f6fa8069f30 2023-07-08T12:38:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.883+0000 7f6fac9a2700 1 -- 172.21.15.12:0/2723859619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa81e41f0 con 0x7f6fa8069f30 2023-07-08T12:38:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.883+0000 7f6f9ffff700 1 -- 172.21.15.12:0/2723859619 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f980119e0 con 0x7f6fa8069f30 2023-07-08T12:38:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.883+0000 7f6fac9a2700 1 -- 172.21.15.12:0/2723859619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa81e47b0 con 0x7f6fa8069f30 2023-07-08T12:38:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.884+0000 7f6fac9a2700 1 -- 172.21.15.12:0/2723859619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa8067590 con 0x7f6fa8069f30 2023-07-08T12:38:30.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.885+0000 7f6f9ffff700 1 -- 172.21.15.12:0/2723859619 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6f98011b40 con 0x7f6fa8069f30 2023-07-08T12:38:30.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.886+0000 7f6f9ffff700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f8806ace0 0x7f6f8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:30.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.886+0000 7f6f9ffff700 1 -- 172.21.15.12:0/2723859619 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6f9808b3a0 con 0x7f6fa8069f30 2023-07-08T12:38:30.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.886+0000 7f6fa6ffd700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f8806ace0 0x7f6f8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:30.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.886+0000 7f6fa6ffd700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f8806ace0 0x7f6f8806d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80b0cf0 tx=0x7f6f90007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:30.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:30.889+0000 7f6f9ffff700 1 -- 172.21.15.12:0/2723859619 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6f980561d0 con 0x7f6fa8069f30 2023-07-08T12:38:31.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.135+0000 7f6fac9a2700 1 -- 172.21.15.12:0/2723859619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6fa806dc90 con 0x7f6fa8069f30 2023-07-08T12:38:31.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.320+0000 7f6f9ffff700 1 -- 172.21.15.12:0/2723859619 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6f980597f0 con 0x7f6fa8069f30 2023-07-08T12:38:31.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.322+0000 7f6fac9a2700 1 -- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f8806ace0 msgr2=0x7f6f8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:31.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.322+0000 7f6fac9a2700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f8806ace0 0x7f6f8806d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80b0cf0 tx=0x7f6f90007ec0 comp rx=0 tx=0).stop 2023-07-08T12:38:31.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.322+0000 7f6fac9a2700 1 -- 172.21.15.12:0/2723859619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa8069f30 msgr2=0x7f6fa814fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:31.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.322+0000 7f6fac9a2700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa8069f30 0x7f6fa814fab0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6f980090b0 tx=0x7f6f980047c0 comp rx=0 tx=0).stop 2023-07-08T12:38:31.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.322+0000 7f6fac9a2700 1 -- 172.21.15.12:0/2723859619 shutdown_connections 2023-07-08T12:38:31.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.322+0000 7f6fac9a2700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6f8806ace0 0x7f6f8806d190 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:31.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.322+0000 7f6fac9a2700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fa81562b0 0x7f6fa80c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:31.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.322+0000 7f6fac9a2700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa8069f30 0x7f6fa814fab0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:31.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.322+0000 7f6fac9a2700 1 --2- 172.21.15.12:0/2723859619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fa80ceca0 0x7f6fa814f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:31.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.322+0000 7f6fac9a2700 1 -- 172.21.15.12:0/2723859619 >> 172.21.15.12:0/2723859619 conn(0x7f6fa806bcd0 msgr2=0x7f6fa80b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:31.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.322+0000 7f6fac9a2700 1 -- 172.21.15.12:0/2723859619 shutdown_connections 2023-07-08T12:38:31.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.322+0000 7f6fac9a2700 1 -- 172.21.15.12:0/2723859619 wait complete. 2023-07-08T12:38:31.336 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-07-08T12:38:31.336 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-08T12:38:31.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.463+0000 7f05a2b90700 1 Processor -- start 2023-07-08T12:38:31.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.463+0000 7f05a2b90700 1 -- start start 2023-07-08T12:38:31.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.463+0000 7f05a2b90700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f059c0aeec0 0x7f059c06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:31.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.463+0000 7f05a2b90700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f059c06ebe0 0x7f059c0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:31.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.463+0000 7f05a2b90700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059c15d770 con 0x7f059c0cd9b0 2023-07-08T12:38:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f05a2b90700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c15a600 con 0x7f059c0aeec0 2023-07-08T12:38:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f05a2b90700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0cddd0 con 0x7f059c06ebe0 2023-07-08T12:38:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f05a092c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f059c0aeec0 0x7f059c06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f05a112d700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f059c0cd9b0 0x7f059c0cbe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36914/0 (socket says 172.21.15.12:36914) 2023-07-08T12:38:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f05a112d700 1 -- 172.21.15.12:0/217936727 learned_addr learned my addr 172.21.15.12:0/217936727 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:31.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f05a092c700 1 -- 172.21.15.12:0/217936727 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f059c06ebe0 msgr2=0x7f059c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:31.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f059bfff700 1 --2- 172.21.15.12:0/217936727 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f059c06ebe0 0x7f059c0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:31.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f05a092c700 1 --2- 172.21.15.12:0/217936727 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f059c06ebe0 0x7f059c0cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:31.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f05a092c700 1 -- 172.21.15.12:0/217936727 >> v1:172.21.15.12:6789/0 conn(0x7f059c0cd9b0 legacy=0x7f059c0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f059b7fe700 1 -- 172.21.15.12:0/217936727 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2382129217 0 0) 0x7f059c15d770 con 0x7f059c0cd9b0 2023-07-08T12:38:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f05a092c700 1 -- 172.21.15.12:0/217936727 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c15cb60 con 0x7f059c0aeec0 2023-07-08T12:38:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f05a092c700 1 --2- 172.21.15.12:0/217936727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f059c0aeec0 0x7f059c06cff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f058c009820 tx=0x7f058c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc680826f24afe3 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f059b7fe700 1 -- 172.21.15.12:0/217936727 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c005e30 con 0x7f059c0aeec0 2023-07-08T12:38:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f059b7fe700 1 -- 172.21.15.12:0/217936727 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c006030 con 0x7f059c0aeec0 2023-07-08T12:38:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.464+0000 7f059b7fe700 1 -- 172.21.15.12:0/217936727 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c0063e0 con 0x7f059c0aeec0 2023-07-08T12:38:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.465+0000 7f05a2b90700 1 -- 172.21.15.12:0/217936727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f059c0aeec0 msgr2=0x7f059c06cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.465+0000 7f05a2b90700 1 --2- 172.21.15.12:0/217936727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f059c0aeec0 0x7f059c06cff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f058c009820 tx=0x7f058c009b30 comp rx=0 tx=0).stop 2023-07-08T12:38:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.465+0000 7f05a2b90700 1 -- 172.21.15.12:0/217936727 shutdown_connections 2023-07-08T12:38:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.465+0000 7f05a2b90700 1 --2- 172.21.15.12:0/217936727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f059c0aeec0 0x7f059c06cff0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.465+0000 7f05a2b90700 1 --2- 172.21.15.12:0/217936727 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f059c06ebe0 0x7f059c0cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.465+0000 7f05a2b90700 1 -- 172.21.15.12:0/217936727 >> 172.21.15.12:0/217936727 conn(0x7f059c098b50 msgr2=0x7f059c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.465+0000 7f05a2b90700 1 -- 172.21.15.12:0/217936727 shutdown_connections 2023-07-08T12:38:31.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.465+0000 7f05a2b90700 1 -- 172.21.15.12:0/217936727 wait complete. 2023-07-08T12:38:31.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.466+0000 7f05a2b90700 1 Processor -- start 2023-07-08T12:38:31.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.466+0000 7f05a2b90700 1 -- start start 2023-07-08T12:38:31.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f05a2b90700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f059c06ebe0 0x7f059c156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:31.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f05a2b90700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f059c0aeec0 0x7f059c14ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:31.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f05a2b90700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f059c0cd9b0 0x7f059c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:31.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f05a2b90700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0c7a40 con 0x7f059c0aeec0 2023-07-08T12:38:31.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f05a2b90700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0c7d20 con 0x7f059c06ebe0 2023-07-08T12:38:31.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f05a2b90700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0c8000 con 0x7f059c0cd9b0 2023-07-08T12:38:31.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f059bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f059c0aeec0 0x7f059c14ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:31.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f059bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f059c0aeec0 0x7f059c14ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49522/0 (socket says 172.21.15.12:49522) 2023-07-08T12:38:31.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f059bfff700 1 -- 172.21.15.12:0/1951859991 learned_addr learned my addr 172.21.15.12:0/1951859991 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:31.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f05a092c700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f059c06ebe0 0x7f059c156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:31.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f05a112d700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f059c0cd9b0 0x7f059c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:31.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f059bfff700 1 -- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f059c0cd9b0 msgr2=0x7f059c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:31.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f059bfff700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f059c0cd9b0 0x7f059c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:31.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f059bfff700 1 -- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f059c06ebe0 msgr2=0x7f059c156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:31.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f059bfff700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f059c06ebe0 0x7f059c156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:31.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.467+0000 7f059bfff700 1 -- 172.21.15.12:0/1951859991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c009430 con 0x7f059c0aeec0 2023-07-08T12:38:31.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.468+0000 7f059bfff700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f059c0aeec0 0x7f059c14ae10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f058400c7f0 tx=0x7f058400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:31.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.468+0000 7f05997fa700 1 -- 172.21.15.12:0/1951859991 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584011800 con 0x7f059c0aeec0 2023-07-08T12:38:31.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.468+0000 7f05997fa700 1 -- 172.21.15.12:0/1951859991 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0584011960 con 0x7f059c0aeec0 2023-07-08T12:38:31.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.468+0000 7f05997fa700 1 -- 172.21.15.12:0/1951859991 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584011cf0 con 0x7f059c0aeec0 2023-07-08T12:38:31.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.468+0000 7f05a2b90700 1 -- 172.21.15.12:0/1951859991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c0c82e0 con 0x7f059c0aeec0 2023-07-08T12:38:31.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.468+0000 7f05a2b90700 1 -- 172.21.15.12:0/1951859991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c0c3290 con 0x7f059c0aeec0 2023-07-08T12:38:31.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.469+0000 7f05a2b90700 1 -- 172.21.15.12:0/1951859991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059c0c3510 con 0x7f059c0aeec0 2023-07-08T12:38:31.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.469+0000 7f05997fa700 1 -- 172.21.15.12:0/1951859991 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0584018360 con 0x7f059c0aeec0 2023-07-08T12:38:31.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.470+0000 7f05997fa700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f057c06ace0 0x7f057c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:31.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.470+0000 7f05997fa700 1 -- 172.21.15.12:0/1951859991 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f05840897a0 con 0x7f059c0aeec0 2023-07-08T12:38:31.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.470+0000 7f05a092c700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f057c06ace0 0x7f057c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:31.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.470+0000 7f05a092c700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f057c06ace0 0x7f057c06d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f058c013040 tx=0x7f058c008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:31.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.474+0000 7f05997fa700 1 -- 172.21.15.12:0/1951859991 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0584058000 con 0x7f059c0aeec0 2023-07-08T12:38:31.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:31.722+0000 7f05a2b90700 1 -- 172.21.15.12:0/1951859991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f059c0c7b80 con 0x7f059c0aeec0 2023-07-08T12:38:32.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.329+0000 7f05997fa700 1 -- 172.21.15.12:0/1951859991 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f059c0c7b80 con 0x7f059c0aeec0 2023-07-08T12:38:32.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.332+0000 7f05a2b90700 1 -- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f057c06ace0 msgr2=0x7f057c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:32.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.332+0000 7f05a2b90700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f057c06ace0 0x7f057c06d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f058c013040 tx=0x7f058c008070 comp rx=0 tx=0).stop 2023-07-08T12:38:32.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.332+0000 7f05a2b90700 1 -- 172.21.15.12:0/1951859991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f059c0aeec0 msgr2=0x7f059c14ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:32.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.332+0000 7f05a2b90700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f059c0aeec0 0x7f059c14ae10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f058400c7f0 tx=0x7f058400cbb0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.332+0000 7f05a2b90700 1 -- 172.21.15.12:0/1951859991 shutdown_connections 2023-07-08T12:38:32.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.332+0000 7f05a2b90700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f057c06ace0 0x7f057c06d190 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.332+0000 7f05a2b90700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f059c0cd9b0 0x7f059c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.332+0000 7f05a2b90700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f059c06ebe0 0x7f059c156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.332+0000 7f05a2b90700 1 --2- 172.21.15.12:0/1951859991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f059c0aeec0 0x7f059c14ae10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.332+0000 7f05a2b90700 1 -- 172.21.15.12:0/1951859991 >> 172.21.15.12:0/1951859991 conn(0x7f059c098b50 msgr2=0x7f059c15b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:32.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.332+0000 7f05a2b90700 1 -- 172.21.15.12:0/1951859991 shutdown_connections 2023-07-08T12:38:32.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.332+0000 7f05a2b90700 1 -- 172.21.15.12:0/1951859991 wait complete. 2023-07-08T12:38:32.348 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-08T12:38:32.349 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-08T12:38:32.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.479+0000 7fc08ac3e700 1 Processor -- start 2023-07-08T12:38:32.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.479+0000 7fc08ac3e700 1 -- start start 2023-07-08T12:38:32.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc08ac3e700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc0840ce1b0 0x7fc0840cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:32.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc08ac3e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0840b1c70 0x7fc0840ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:32.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc08ac3e700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08415d230 con 0x7fc08415d6c0 2023-07-08T12:38:32.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc08ac3e700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0840c9e60 con 0x7fc0840ce1b0 2023-07-08T12:38:32.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc08ac3e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840cd920 con 0x7fc0840b1c70 2023-07-08T12:38:32.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc0889da700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc0840ce1b0 0x7fc0840cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:32.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc0891db700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc08415d6c0 0x7fc084071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36928/0 (socket says 172.21.15.12:36928) 2023-07-08T12:38:32.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc0891db700 1 -- 172.21.15.12:0/674823976 learned_addr learned my addr 172.21.15.12:0/674823976 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:32.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc083fff700 1 --2- 172.21.15.12:0/674823976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0840b1c70 0x7fc0840ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:32.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc0889da700 1 -- 172.21.15.12:0/674823976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0840b1c70 msgr2=0x7fc0840ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:32.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc0889da700 1 --2- 172.21.15.12:0/674823976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0840b1c70 0x7fc0840ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc0889da700 1 -- 172.21.15.12:0/674823976 >> v1:172.21.15.12:6789/0 conn(0x7fc08415d6c0 legacy=0x7fc084071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:32.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc0889da700 1 -- 172.21.15.12:0/674823976 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0840ceac0 con 0x7fc0840ce1b0 2023-07-08T12:38:32.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.480+0000 7fc0889da700 1 --2- 172.21.15.12:0/674823976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0840ce1b0 0x7fc0840cd3e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc07800bae0 tx=0x7fc07800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef70a24ad99f126c server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:32.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.481+0000 7fc0837fe700 1 -- 172.21.15.12:0/674823976 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800dc30 con 0x7fc0840ce1b0 2023-07-08T12:38:32.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.481+0000 7fc0837fe700 1 -- 172.21.15.12:0/674823976 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07800dd90 con 0x7fc0840ce1b0 2023-07-08T12:38:32.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.481+0000 7fc0837fe700 1 -- 172.21.15.12:0/674823976 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078004250 con 0x7fc0840ce1b0 2023-07-08T12:38:32.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.481+0000 7fc08ac3e700 1 -- 172.21.15.12:0/674823976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0840ce1b0 msgr2=0x7fc0840cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:32.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.481+0000 7fc08ac3e700 1 --2- 172.21.15.12:0/674823976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0840ce1b0 0x7fc0840cd3e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc07800bae0 tx=0x7fc07800bdf0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.481+0000 7fc08ac3e700 1 -- 172.21.15.12:0/674823976 shutdown_connections 2023-07-08T12:38:32.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.481+0000 7fc08ac3e700 1 --2- 172.21.15.12:0/674823976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0840ce1b0 0x7fc0840cd3e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.481+0000 7fc08ac3e700 1 --2- 172.21.15.12:0/674823976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0840b1c70 0x7fc0840ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.481+0000 7fc08ac3e700 1 -- 172.21.15.12:0/674823976 >> 172.21.15.12:0/674823976 conn(0x7fc0840cc290 msgr2=0x7fc0840cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:32.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.481+0000 7fc08ac3e700 1 -- 172.21.15.12:0/674823976 shutdown_connections 2023-07-08T12:38:32.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.481+0000 7fc08ac3e700 1 -- 172.21.15.12:0/674823976 wait complete. 2023-07-08T12:38:32.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.482+0000 7fc08ac3e700 1 Processor -- start 2023-07-08T12:38:32.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.482+0000 7fc08ac3e700 1 -- start start 2023-07-08T12:38:32.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.483+0000 7fc08ac3e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0840b1c70 0x7fc0840b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:32.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.483+0000 7fc08ac3e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0840ce1b0 0x7fc0840b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:32.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.483+0000 7fc08ac3e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08415d6c0 0x7fc08415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:32.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.483+0000 7fc08ac3e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840b4de0 con 0x7fc08415d6c0 2023-07-08T12:38:32.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.483+0000 7fc08ac3e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08415b7e0 con 0x7fc0840ce1b0 2023-07-08T12:38:32.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.483+0000 7fc08ac3e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840b4160 con 0x7fc0840b1c70 2023-07-08T12:38:32.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.483+0000 7fc0891db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08415d6c0 0x7fc08415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:32.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.483+0000 7fc0891db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08415d6c0 0x7fc08415b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49538/0 (socket says 172.21.15.12:49538) 2023-07-08T12:38:32.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.483+0000 7fc0891db700 1 -- 172.21.15.12:0/2217033685 learned_addr learned my addr 172.21.15.12:0/2217033685 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.484+0000 7fc0889da700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0840b1c70 0x7fc0840b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.484+0000 7fc083fff700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0840ce1b0 0x7fc0840b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.484+0000 7fc0891db700 1 -- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0840b1c70 msgr2=0x7fc0840b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.484+0000 7fc0891db700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0840b1c70 0x7fc0840b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.484+0000 7fc0891db700 1 -- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0840ce1b0 msgr2=0x7fc0840b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.484+0000 7fc0891db700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0840ce1b0 0x7fc0840b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.484+0000 7fc0891db700 1 -- 172.21.15.12:0/2217033685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07800b740 con 0x7fc08415d6c0 2023-07-08T12:38:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.484+0000 7fc0891db700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08415d6c0 0x7fc08415b2a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc07c002000 tx=0x7fc07c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.484+0000 7fc0817fa700 1 -- 172.21.15.12:0/2217033685 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00ac90 con 0x7fc08415d6c0 2023-07-08T12:38:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.484+0000 7fc0817fa700 1 -- 172.21.15.12:0/2217033685 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c00adf0 con 0x7fc08415d6c0 2023-07-08T12:38:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.484+0000 7fc08ac3e700 1 -- 172.21.15.12:0/2217033685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0840b4440 con 0x7fc08415d6c0 2023-07-08T12:38:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.484+0000 7fc0817fa700 1 -- 172.21.15.12:0/2217033685 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c007310 con 0x7fc08415d6c0 2023-07-08T12:38:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.484+0000 7fc08ac3e700 1 -- 172.21.15.12:0/2217033685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0840b4a00 con 0x7fc08415d6c0 2023-07-08T12:38:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.485+0000 7fc08ac3e700 1 -- 172.21.15.12:0/2217033685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0840b4f20 con 0x7fc08415d6c0 2023-07-08T12:38:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.486+0000 7fc0817fa700 1 -- 172.21.15.12:0/2217033685 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc07c007470 con 0x7fc08415d6c0 2023-07-08T12:38:32.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.486+0000 7fc0817fa700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc06806ace0 0x7fc06806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.486+0000 7fc0817fa700 1 -- 172.21.15.12:0/2217033685 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc07c089650 con 0x7fc08415d6c0 2023-07-08T12:38:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.486+0000 7fc0889da700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc06806ace0 0x7fc06806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:32.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.487+0000 7fc0889da700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc06806ace0 0x7fc06806d190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc07800b2c0 tx=0x7fc078013000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:32.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.490+0000 7fc0817fa700 1 -- 172.21.15.12:0/2217033685 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc0840b4f20 con 0x7fc08415d6c0 2023-07-08T12:38:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.741+0000 7fc08ac3e700 1 -- 172.21.15.12:0/2217033685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0840b4f20 con 0x7fc08415d6c0 2023-07-08T12:38:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.741+0000 7fc0817fa700 1 -- 172.21.15.12:0/2217033685 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fc0840b4f20 con 0x7fc08415d6c0 2023-07-08T12:38:32.743 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:32.743 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3238006310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3238006310},{"type":"v1","addr":"172.21.15.160:6836","nonce":3238006310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3864661454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3864661454},{"type":"v1","addr":"172.21.15.160:6837","nonce":3864661454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6834/946118830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":946118830},{"type":"v1","addr":"172.21.15.12:6834","nonce":946118830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-08T12:38:07.943202+0000","modified":"2023-07-08T12:38:32.319568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6835/3301134992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3301134992},{"type":"v1","addr":"172.21.15.12:6835","nonce":3301134992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-08T12:38:32.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.743+0000 7fc08ac3e700 1 -- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc06806ace0 msgr2=0x7fc06806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:32.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.743+0000 7fc08ac3e700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc06806ace0 0x7fc06806d190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc07800b2c0 tx=0x7fc078013000 comp rx=0 tx=0).stop 2023-07-08T12:38:32.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.743+0000 7fc08ac3e700 1 -- 172.21.15.12:0/2217033685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08415d6c0 msgr2=0x7fc08415b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:32.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.743+0000 7fc08ac3e700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08415d6c0 0x7fc08415b2a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc07c002000 tx=0x7fc07c00a340 comp rx=0 tx=0).stop 2023-07-08T12:38:32.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.743+0000 7fc08ac3e700 1 -- 172.21.15.12:0/2217033685 shutdown_connections 2023-07-08T12:38:32.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.743+0000 7fc08ac3e700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc06806ace0 0x7fc06806d190 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.743+0000 7fc08ac3e700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc08415d6c0 0x7fc08415b2a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.743+0000 7fc08ac3e700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0840b1c70 0x7fc0840b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.743+0000 7fc08ac3e700 1 --2- 172.21.15.12:0/2217033685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0840ce1b0 0x7fc0840b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.743+0000 7fc08ac3e700 1 -- 172.21.15.12:0/2217033685 >> 172.21.15.12:0/2217033685 conn(0x7fc0840cc290 msgr2=0x7fc0840cb260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.743+0000 7fc08ac3e700 1 -- 172.21.15.12:0/2217033685 shutdown_connections 2023-07-08T12:38:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.744+0000 7fc08ac3e700 1 -- 172.21.15.12:0/2217033685 wait complete. 2023-07-08T12:38:32.747 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2023-07-08T12:38:32.757 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:32.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.880+0000 7fac78b8c700 1 Processor -- start 2023-07-08T12:38:32.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.880+0000 7fac78b8c700 1 -- start start 2023-07-08T12:38:32.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac78b8c700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fac7415a480 0x7fac740cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:32.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac78b8c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac740cfea0 0x7fac740b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:32.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac78b8c700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7415a850 con 0x7fac740b2410 2023-07-08T12:38:32.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac78b8c700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac740b2da0 con 0x7fac7415a480 2023-07-08T12:38:32.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac78b8c700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac740cde90 con 0x7fac740cfea0 2023-07-08T12:38:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac71d9b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fac740b2410 0x7fac7415def0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36940/0 (socket says 172.21.15.12:36940) 2023-07-08T12:38:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac71d9b700 1 -- 172.21.15.12:0/1079378568 learned_addr learned my addr 172.21.15.12:0/1079378568 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac72d9d700 1 --2- 172.21.15.12:0/1079378568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac740cfea0 0x7fac740b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac7259c700 1 --2- 172.21.15.12:0/1079378568 >> v2:172.21.15.160:3300/0 conn(0x7fac7415a480 0x7fac740cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac7159a700 1 -- 172.21.15.12:0/1079378568 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892521718 0 0) 0x7fac68001110 con 0x7fac740b2410 2023-07-08T12:38:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac7159a700 1 -- 172.21.15.12:0/1079378568 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac5c0035b0 con 0x7fac740b2410 2023-07-08T12:38:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac72d9d700 1 -- 172.21.15.12:0/1079378568 >> v2:172.21.15.160:3300/0 conn(0x7fac7415a480 msgr2=0x7fac740cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac72d9d700 1 --2- 172.21.15.12:0/1079378568 >> v2:172.21.15.160:3300/0 conn(0x7fac7415a480 0x7fac740cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac72d9d700 1 -- 172.21.15.12:0/1079378568 >> v1:172.21.15.12:6789/0 conn(0x7fac740b2410 legacy=0x7fac7415def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac72d9d700 1 -- 172.21.15.12:0/1079378568 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac740cd8d0 con 0x7fac740cfea0 2023-07-08T12:38:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.881+0000 7fac7259c700 1 --2- 172.21.15.12:0/1079378568 >> v2:172.21.15.160:3300/0 conn(0x7fac7415a480 0x7fac740cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:38:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.882+0000 7fac72d9d700 1 --2- 172.21.15.12:0/1079378568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac740cfea0 0x7fac740b2860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009d90 tx=0x7fac6c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e792669cdadb98d server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.882+0000 7fac7159a700 1 -- 172.21.15.12:0/1079378568 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c00e070 con 0x7fac740cfea0 2023-07-08T12:38:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.882+0000 7fac7159a700 1 -- 172.21.15.12:0/1079378568 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c00bb20 con 0x7fac740cfea0 2023-07-08T12:38:32.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.882+0000 7fac7159a700 1 -- 172.21.15.12:0/1079378568 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c00be90 con 0x7fac740cfea0 2023-07-08T12:38:32.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.882+0000 7fac78b8c700 1 -- 172.21.15.12:0/1079378568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac740cfea0 msgr2=0x7fac740b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:32.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.882+0000 7fac78b8c700 1 --2- 172.21.15.12:0/1079378568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac740cfea0 0x7fac740b2860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009d90 tx=0x7fac6c00b330 comp rx=0 tx=0).stop 2023-07-08T12:38:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.882+0000 7fac78b8c700 1 -- 172.21.15.12:0/1079378568 shutdown_connections 2023-07-08T12:38:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.882+0000 7fac78b8c700 1 --2- 172.21.15.12:0/1079378568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac740cfea0 0x7fac740b2860 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.882+0000 7fac78b8c700 1 --2- 172.21.15.12:0/1079378568 >> v2:172.21.15.160:3300/0 conn(0x7fac7415a480 0x7fac740cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.882+0000 7fac78b8c700 1 -- 172.21.15.12:0/1079378568 >> 172.21.15.12:0/1079378568 conn(0x7fac7406ab60 msgr2=0x7fac7406e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.882+0000 7fac78b8c700 1 -- 172.21.15.12:0/1079378568 shutdown_connections 2023-07-08T12:38:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.882+0000 7fac78b8c700 1 -- 172.21.15.12:0/1079378568 wait complete. 2023-07-08T12:38:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.884+0000 7fac78b8c700 1 Processor -- start 2023-07-08T12:38:32.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.884+0000 7fac78b8c700 1 -- start start 2023-07-08T12:38:32.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac78b8c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac740b2410 0x7fac740b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:32.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac78b8c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac740cfea0 0x7fac740c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:32.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac78b8c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac7415a480 0x7fac740c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:32.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac78b8c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac741534a0 con 0x7fac740cfea0 2023-07-08T12:38:32.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac78b8c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac740b61d0 con 0x7fac740b2410 2023-07-08T12:38:32.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac78b8c700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac740b4bf0 con 0x7fac7415a480 2023-07-08T12:38:32.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac71d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac740cfea0 0x7fac740c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:32.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac71d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac740cfea0 0x7fac740c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49550/0 (socket says 172.21.15.12:49550) 2023-07-08T12:38:32.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac71d9b700 1 -- 172.21.15.12:0/141503246 learned_addr learned my addr 172.21.15.12:0/141503246 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:32.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac7259c700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac740b2410 0x7fac740b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:32.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac72d9d700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac7415a480 0x7fac740c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:32.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac71d9b700 1 -- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac7415a480 msgr2=0x7fac740c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:32.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac71d9b700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac7415a480 0x7fac740c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac71d9b700 1 -- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac740b2410 msgr2=0x7fac740b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:32.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac71d9b700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac740b2410 0x7fac740b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:32.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac71d9b700 1 -- 172.21.15.12:0/141503246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac640094a0 con 0x7fac740cfea0 2023-07-08T12:38:32.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac71d9b700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac740cfea0 0x7fac740c7a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fac6800c420 tx=0x7fac6800c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:32.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac62ffd700 1 -- 172.21.15.12:0/141503246 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68005d70 con 0x7fac740cfea0 2023-07-08T12:38:32.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac62ffd700 1 -- 172.21.15.12:0/141503246 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6800ce30 con 0x7fac740cfea0 2023-07-08T12:38:32.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac78b8c700 1 -- 172.21.15.12:0/141503246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6c009a70 con 0x7fac740cfea0 2023-07-08T12:38:32.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac62ffd700 1 -- 172.21.15.12:0/141503246 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68007310 con 0x7fac740cfea0 2023-07-08T12:38:32.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.885+0000 7fac78b8c700 1 -- 172.21.15.12:0/141503246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac740b51b0 con 0x7fac740cfea0 2023-07-08T12:38:32.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.886+0000 7fac78b8c700 1 -- 172.21.15.12:0/141503246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7404ea60 con 0x7fac740cfea0 2023-07-08T12:38:32.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.887+0000 7fac62ffd700 1 -- 172.21.15.12:0/141503246 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fac68007470 con 0x7fac740cfea0 2023-07-08T12:38:32.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.887+0000 7fac62ffd700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac5406ace0 0x7fac5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:32.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.887+0000 7fac62ffd700 1 -- 172.21.15.12:0/141503246 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fac680889f0 con 0x7fac740cfea0 2023-07-08T12:38:32.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.888+0000 7fac7259c700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac5406ace0 0x7fac5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:32.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.888+0000 7fac7259c700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac5406ace0 0x7fac5406d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fac740b68c0 tx=0x7fac64009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:32.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:32.892+0000 7fac62ffd700 1 -- 172.21.15.12:0/141503246 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fac68053710 con 0x7fac740cfea0 2023-07-08T12:38:33.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.094+0000 7fac78b8c700 1 -- 172.21.15.12:0/141503246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac7415a850 con 0x7fac740cfea0 2023-07-08T12:38:33.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.095+0000 7fac62ffd700 1 -- 172.21.15.12:0/141503246 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20108 (secure 0 0 0) 0x7fac7415a850 con 0x7fac740cfea0 2023-07-08T12:38:33.097 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:33.097 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:12.993803+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:33.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.097+0000 7fac78b8c700 1 -- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac5406ace0 msgr2=0x7fac5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.097+0000 7fac78b8c700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac5406ace0 0x7fac5406d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fac740b68c0 tx=0x7fac64009210 comp rx=0 tx=0).stop 2023-07-08T12:38:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.097+0000 7fac78b8c700 1 -- 172.21.15.12:0/141503246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac740cfea0 msgr2=0x7fac740c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.097+0000 7fac78b8c700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac740cfea0 0x7fac740c7a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fac6800c420 tx=0x7fac6800c730 comp rx=0 tx=0).stop 2023-07-08T12:38:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.097+0000 7fac78b8c700 1 -- 172.21.15.12:0/141503246 shutdown_connections 2023-07-08T12:38:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.097+0000 7fac78b8c700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac5406ace0 0x7fac5406d190 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.097+0000 7fac78b8c700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac7415a480 0x7fac740c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.097+0000 7fac78b8c700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac740b2410 0x7fac740b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.097+0000 7fac78b8c700 1 --2- 172.21.15.12:0/141503246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac740cfea0 0x7fac740c7a60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.097+0000 7fac78b8c700 1 -- 172.21.15.12:0/141503246 >> 172.21.15.12:0/141503246 conn(0x7fac7406ab60 msgr2=0x7fac740b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:33.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.097+0000 7fac78b8c700 1 -- 172.21.15.12:0/141503246 shutdown_connections 2023-07-08T12:38:33.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.097+0000 7fac78b8c700 1 -- 172.21.15.12:0/141503246 wait complete. 2023-07-08T12:38:33.111 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-08T12:38:33.111 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-08T12:38:33.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.235+0000 7f9da06bb700 1 Processor -- start 2023-07-08T12:38:33.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.235+0000 7f9da06bb700 1 -- start start 2023-07-08T12:38:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.236+0000 7f9da06bb700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9d98070bb0 0x7f9d9815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.236+0000 7f9da06bb700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9806b1b0 con 0x7f9d980cd530 2023-07-08T12:38:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.236+0000 7f9da06bb700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9815d1f0 con 0x7f9d9806b640 2023-07-08T12:38:33.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.236+0000 7f9da06bb700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d9806a5a0 con 0x7f9d98070bb0 2023-07-08T12:38:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.236+0000 7f9d9dc56700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9d980cd530 0x7f9d98070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36954/0 (socket says 172.21.15.12:36954) 2023-07-08T12:38:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.236+0000 7f9d9dc56700 1 -- 172.21.15.12:0/982967019 learned_addr learned my addr 172.21.15.12:0/982967019 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.236+0000 7f9d9e457700 1 --2- 172.21.15.12:0/982967019 >> v2:172.21.15.160:3300/0 conn(0x7f9d98070bb0 0x7f9d9815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.236+0000 7f9d9d455700 1 -- 172.21.15.12:0/982967019 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4259143985 0 0) 0x7f9d9806b1b0 con 0x7f9d980cd530 2023-07-08T12:38:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.236+0000 7f9d9d455700 1 -- 172.21.15.12:0/982967019 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d840035b0 con 0x7f9d980cd530 2023-07-08T12:38:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.236+0000 7f9d9e457700 1 -- 172.21.15.12:0/982967019 >> v1:172.21.15.160:6789/0 conn(0x7f9d9806b640 legacy=0x7f9d980c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.236+0000 7f9d9e457700 1 -- 172.21.15.12:0/982967019 >> v1:172.21.15.12:6789/0 conn(0x7f9d980cd530 legacy=0x7f9d98070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.236+0000 7f9d9e457700 1 -- 172.21.15.12:0/982967019 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9806a880 con 0x7f9d98070bb0 2023-07-08T12:38:33.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.237+0000 7f9d9e457700 1 --2- 172.21.15.12:0/982967019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d98070bb0 0x7f9d9815ccb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009840 tx=0x7f9d88009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caa208307d0a8fdc server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:33.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.237+0000 7f9d9d455700 1 -- 172.21.15.12:0/982967019 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88011830 con 0x7f9d98070bb0 2023-07-08T12:38:33.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.237+0000 7f9d9d455700 1 -- 172.21.15.12:0/982967019 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d88011990 con 0x7f9d98070bb0 2023-07-08T12:38:33.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.237+0000 7f9d9d455700 1 -- 172.21.15.12:0/982967019 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88011d20 con 0x7f9d98070bb0 2023-07-08T12:38:33.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.237+0000 7f9da06bb700 1 -- 172.21.15.12:0/982967019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d98070bb0 msgr2=0x7f9d9815ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.237+0000 7f9da06bb700 1 --2- 172.21.15.12:0/982967019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d98070bb0 0x7f9d9815ccb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009840 tx=0x7f9d88009b50 comp rx=0 tx=0).stop 2023-07-08T12:38:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.237+0000 7f9da06bb700 1 -- 172.21.15.12:0/982967019 shutdown_connections 2023-07-08T12:38:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.237+0000 7f9da06bb700 1 --2- 172.21.15.12:0/982967019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d98070bb0 0x7f9d9815ccb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.237+0000 7f9da06bb700 1 -- 172.21.15.12:0/982967019 >> 172.21.15.12:0/982967019 conn(0x7f9d980cb9f0 msgr2=0x7f9d980cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.237+0000 7f9da06bb700 1 -- 172.21.15.12:0/982967019 shutdown_connections 2023-07-08T12:38:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.237+0000 7f9da06bb700 1 -- 172.21.15.12:0/982967019 wait complete. 2023-07-08T12:38:33.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.239+0000 7f9da06bb700 1 Processor -- start 2023-07-08T12:38:33.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.239+0000 7f9da06bb700 1 -- start start 2023-07-08T12:38:33.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.239+0000 7f9da06bb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d9806b640 0x7f9d980a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:33.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.239+0000 7f9da06bb700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9d98070bb0 0x7f9d980c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:33.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.239+0000 7f9da06bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d980cd530 0x7f9d980c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:33.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.239+0000 7f9da06bb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d980c7760 con 0x7f9d980cd530 2023-07-08T12:38:33.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.239+0000 7f9da06bb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d980a79a0 con 0x7f9d9806b640 2023-07-08T12:38:33.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.239+0000 7f9da06bb700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d980a7c80 con 0x7f9d98070bb0 2023-07-08T12:38:33.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d9ec58700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d980cd530 0x7f9d980c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:33.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d9ec58700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d980cd530 0x7f9d980c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49566/0 (socket says 172.21.15.12:49566) 2023-07-08T12:38:33.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d9ec58700 1 -- 172.21.15.12:0/3360714020 learned_addr learned my addr 172.21.15.12:0/3360714020 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:33.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d9dc56700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9d98070bb0 0x7f9d980c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:33.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d9e457700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d9806b640 0x7f9d980a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:33.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d9ec58700 1 -- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9d98070bb0 msgr2=0x7f9d980c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d9ec58700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9d98070bb0 0x7f9d980c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d9ec58700 1 -- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d9806b640 msgr2=0x7f9d980a94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d9ec58700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d9806b640 0x7f9d980a94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d9ec58700 1 -- 172.21.15.12:0/3360714020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d880094a0 con 0x7f9d980cd530 2023-07-08T12:38:33.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d9ec58700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d980cd530 0x7f9d980c8520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000cb50 tx=0x7f9d9000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:33.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d8effd700 1 -- 172.21.15.12:0/3360714020 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90003c00 con 0x7f9d980cd530 2023-07-08T12:38:33.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d8effd700 1 -- 172.21.15.12:0/3360714020 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d90003d60 con 0x7f9d980cd530 2023-07-08T12:38:33.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9da06bb700 1 -- 172.21.15.12:0/3360714020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d980a7f60 con 0x7f9d980cd530 2023-07-08T12:38:33.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9d8effd700 1 -- 172.21.15.12:0/3360714020 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000ab70 con 0x7f9d980cd530 2023-07-08T12:38:33.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.240+0000 7f9da06bb700 1 -- 172.21.15.12:0/3360714020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d980a8520 con 0x7f9d980cd530 2023-07-08T12:38:33.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.241+0000 7f9da06bb700 1 -- 172.21.15.12:0/3360714020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9804ea60 con 0x7f9d980cd530 2023-07-08T12:38:33.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.242+0000 7f9d8effd700 1 -- 172.21.15.12:0/3360714020 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d9000acd0 con 0x7f9d980cd530 2023-07-08T12:38:33.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.242+0000 7f9d8effd700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9d7c06ace0 0x7f9d7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:33.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.242+0000 7f9d8effd700 1 -- 172.21.15.12:0/3360714020 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9d900897c0 con 0x7f9d980cd530 2023-07-08T12:38:33.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.242+0000 7f9d9e457700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9d7c06ace0 0x7f9d7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:33.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.243+0000 7f9d9e457700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9d7c06ace0 0x7f9d7c06d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d880090b0 tx=0x7f9d88008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.247+0000 7f9d8effd700 1 -- 172.21.15.12:0/3360714020 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9d90054590 con 0x7f9d980cd530 2023-07-08T12:38:33.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.442+0000 7f9da06bb700 1 -- 172.21.15.12:0/3360714020 --> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9d9806ae40 con 0x7f9d7c06ace0 2023-07-08T12:38:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.453+0000 7f9d8effd700 1 -- 172.21.15.12:0/3360714020 <== mgr.4103 v2:172.21.15.160:6832/550875184 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367308 (secure 0 0 0) 0x7f9d88011b00 con 0x7f9d7c06ace0 2023-07-08T12:38:33.458 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.456+0000 7f9da06bb700 1 -- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9d7c06ace0 msgr2=0x7f9d7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.456+0000 7f9da06bb700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9d7c06ace0 0x7f9d7c06d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d880090b0 tx=0x7f9d88008030 comp rx=0 tx=0).stop 2023-07-08T12:38:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.456+0000 7f9da06bb700 1 -- 172.21.15.12:0/3360714020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d980cd530 msgr2=0x7f9d980c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.456+0000 7f9da06bb700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d980cd530 0x7f9d980c8520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000cb50 tx=0x7f9d9000cf10 comp rx=0 tx=0).stop 2023-07-08T12:38:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.456+0000 7f9da06bb700 1 -- 172.21.15.12:0/3360714020 shutdown_connections 2023-07-08T12:38:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.456+0000 7f9da06bb700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9d7c06ace0 0x7f9d7c06d190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.456+0000 7f9da06bb700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d980cd530 0x7f9d980c8520 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.456+0000 7f9da06bb700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d9806b640 0x7f9d980a94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.456+0000 7f9da06bb700 1 --2- 172.21.15.12:0/3360714020 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9d98070bb0 0x7f9d980c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.456+0000 7f9da06bb700 1 -- 172.21.15.12:0/3360714020 >> 172.21.15.12:0/3360714020 conn(0x7f9d980cb9f0 msgr2=0x7f9d980ca620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.457+0000 7f9da06bb700 1 -- 172.21.15.12:0/3360714020 shutdown_connections 2023-07-08T12:38:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.457+0000 7f9da06bb700 1 -- 172.21.15.12:0/3360714020 wait complete. 2023-07-08T12:38:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-07-08T12:38:33.470 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-07-08T12:38:32.145661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594960,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219208,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755512,"statfs":{"total":773094113280,"available":772869644288,"internally_reserved":0,"allocated":3678208,"data_stored":2186496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003442"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604481+0000","last_change":"2023-07-08T12:38:08.558253+0000","last_active":"2023-07-08T12:38:13.604481+0000","last_peered":"2023-07-08T12:38:13.604481+0000","last_clean":"2023-07-08T12:38:13.604481+0000","last_became_active":"2023-07-08T12:38:08.557970+0000","last_became_peered":"2023-07-08T12:38:08.557970+0000","last_unstale":"2023-07-08T12:38:13.604481+0000","last_undegraded":"2023-07-08T12:38:13.604481+0000","last_fullsized":"2023-07-08T12:38:13.604481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:56:17.613729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.608666+0000","last_change":"2023-07-08T12:38:08.546715+0000","last_active":"2023-07-08T12:38:13.608666+0000","last_peered":"2023-07-08T12:38:13.608666+0000","last_clean":"2023-07-08T12:38:13.608666+0000","last_became_active":"2023-07-08T12:38:08.546365+0000","last_became_peered":"2023-07-08T12:38:08.546365+0000","last_unstale":"2023-07-08T12:38:13.608666+0000","last_undegraded":"2023-07-08T12:38:13.608666+0000","last_fullsized":"2023-07-08T12:38:13.608666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:55:50.492319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606777+0000","last_change":"2023-07-08T12:38:08.561903+0000","last_active":"2023-07-08T12:38:13.606777+0000","last_peered":"2023-07-08T12:38:13.606777+0000","last_clean":"2023-07-08T12:38:13.606777+0000","last_became_active":"2023-07-08T12:38:08.561533+0000","last_became_peered":"2023-07-08T12:38:08.561533+0000","last_unstale":"2023-07-08T12:38:13.606777+0000","last_undegraded":"2023-07-08T12:38:13.606777+0000","last_fullsized":"2023-07-08T12:38:13.606777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:26:30.519413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758858+0000","last_change":"2023-07-08T12:38:08.556755+0000","last_active":"2023-07-08T12:38:13.758858+0000","last_peered":"2023-07-08T12:38:13.758858+0000","last_clean":"2023-07-08T12:38:13.758858+0000","last_became_active":"2023-07-08T12:38:08.556476+0000","last_became_peered":"2023-07-08T12:38:08.556476+0000","last_unstale":"2023-07-08T12:38:13.758858+0000","last_undegraded":"2023-07-08T12:38:13.758858+0000","last_fullsized":"2023-07-08T12:38:13.758858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:52.854234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759575+0000","last_change":"2023-07-08T12:38:08.561313+0000","last_active":"2023-07-08T12:38:13.759575+0000","last_peered":"2023-07-08T12:38:13.759575+0000","last_clean":"2023-07-08T12:38:13.759575+0000","last_became_active":"2023-07-08T12:38:08.561055+0000","last_became_peered":"2023-07-08T12:38:08.561055+0000","last_unstale":"2023-07-08T12:38:13.759575+0000","last_undegraded":"2023-07-08T12:38:13.759575+0000","last_fullsized":"2023-07-08T12:38:13.759575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:49.502059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606630+0000","last_change":"2023-07-08T12:38:08.564651+0000","last_active":"2023-07-08T12:38:13.606630+0000","last_peered":"2023-07-08T12:38:13.606630+0000","last_clean":"2023-07-08T12:38:13.606630+0000","last_became_active":"2023-07-08T12:38:08.564389+0000","last_became_peered":"2023-07-08T12:38:08.564389+0000","last_unstale":"2023-07-08T12:38:13.606630+0000","last_undegraded":"2023-07-08T12:38:13.606630+0000","last_fullsized":"2023-07-08T12:38:13.606630+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:03:39.903947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758619+0000","last_change":"2023-07-08T12:38:08.569425+0000","last_active":"2023-07-08T12:38:13.758619+0000","last_peered":"2023-07-08T12:38:13.758619+0000","last_clean":"2023-07-08T12:38:13.758619+0000","last_became_active":"2023-07-08T12:38:08.569167+0000","last_became_peered":"2023-07-08T12:38:08.569167+0000","last_unstale":"2023-07-08T12:38:13.758619+0000","last_undegraded":"2023-07-08T12:38:13.758619+0000","last_fullsized":"2023-07-08T12:38:13.758619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:07.081181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605308+0000","last_change":"2023-07-08T12:38:08.549000+0000","last_active":"2023-07-08T12:38:13.605308+0000","last_peered":"2023-07-08T12:38:13.605308+0000","last_clean":"2023-07-08T12:38:13.605308+0000","last_became_active":"2023-07-08T12:38:08.548657+0000","last_became_peered":"2023-07-08T12:38:08.548657+0000","last_unstale":"2023-07-08T12:38:13.605308+0000","last_undegraded":"2023-07-08T12:38:13.605308+0000","last_fullsized":"2023-07-08T12:38:13.605308+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:37.290100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759490+0000","last_change":"2023-07-08T12:38:08.565002+0000","last_active":"2023-07-08T12:38:13.759490+0000","last_peered":"2023-07-08T12:38:13.759490+0000","last_clean":"2023-07-08T12:38:13.759490+0000","last_became_active":"2023-07-08T12:38:08.564554+0000","last_became_peered":"2023-07-08T12:38:08.564554+0000","last_unstale":"2023-07-08T12:38:13.759490+0000","last_undegraded":"2023-07-08T12:38:13.759490+0000","last_fullsized":"2023-07-08T12:38:13.759490+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:35.758126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606670+0000","last_change":"2023-07-08T12:38:08.556118+0000","last_active":"2023-07-08T12:38:13.606670+0000","last_peered":"2023-07-08T12:38:13.606670+0000","last_clean":"2023-07-08T12:38:13.606670+0000","last_became_active":"2023-07-08T12:38:08.555833+0000","last_became_peered":"2023-07-08T12:38:08.555833+0000","last_unstale":"2023-07-08T12:38:13.606670+0000","last_undegraded":"2023-07-08T12:38:13.606670+0000","last_fullsized":"2023-07-08T12:38:13.606670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:03.390028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606704+0000","last_change":"2023-07-08T12:38:08.568872+0000","last_active":"2023-07-08T12:38:13.606704+0000","last_peered":"2023-07-08T12:38:13.606704+0000","last_clean":"2023-07-08T12:38:13.606704+0000","last_became_active":"2023-07-08T12:38:08.568624+0000","last_became_peered":"2023-07-08T12:38:08.568624+0000","last_unstale":"2023-07-08T12:38:13.606704+0000","last_undegraded":"2023-07-08T12:38:13.606704+0000","last_fullsized":"2023-07-08T12:38:13.606704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:35:09.678657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604156+0000","last_change":"2023-07-08T12:38:08.560950+0000","last_active":"2023-07-08T12:38:13.604156+0000","last_peered":"2023-07-08T12:38:13.604156+0000","last_clean":"2023-07-08T12:38:13.604156+0000","last_became_active":"2023-07-08T12:38:08.560621+0000","last_became_peered":"2023-07-08T12:38:08.560621+0000","last_unstale":"2023-07-08T12:38:13.604156+0000","last_undegraded":"2023-07-08T12:38:13.604156+0000","last_fullsized":"2023-07-08T12:38:13.604156+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T12:59:32.693976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604193+0000","last_change":"2023-07-08T12:38:08.556685+0000","last_active":"2023-07-08T12:38:13.604193+0000","last_peered":"2023-07-08T12:38:13.604193+0000","last_clean":"2023-07-08T12:38:13.604193+0000","last_became_active":"2023-07-08T12:38:08.556387+0000","last_became_peered":"2023-07-08T12:38:08.556387+0000","last_unstale":"2023-07-08T12:38:13.604193+0000","last_undegraded":"2023-07-08T12:38:13.604193+0000","last_fullsized":"2023-07-08T12:38:13.604193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:03:36.626338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604952+0000","last_change":"2023-07-08T12:38:08.547070+0000","last_active":"2023-07-08T12:38:13.604952+0000","last_peered":"2023-07-08T12:38:13.604952+0000","last_clean":"2023-07-08T12:38:13.604952+0000","last_became_active":"2023-07-08T12:38:08.546813+0000","last_became_peered":"2023-07-08T12:38:08.546813+0000","last_unstale":"2023-07-08T12:38:13.604952+0000","last_undegraded":"2023-07-08T12:38:13.604952+0000","last_fullsized":"2023-07-08T12:38:13.604952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758137+0000","last_change":"2023-07-08T12:38:08.569414+0000","last_active":"2023-07-08T12:38:13.758137+0000","last_peered":"2023-07-08T12:38:13.758137+0000","last_clean":"2023-07-08T12:38:13.758137+0000","last_became_active":"2023-07-08T12:38:08.569177+0000","last_became_peered":"2023-07-08T12:38:08.569177+0000","last_unstale":"2023-07-08T12:38:13.758137+0000","last_undegraded":"2023-07-08T12:38:13.758137+0000","last_fullsized":"2023-07-08T12:38:13.758137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606000+0000","last_change":"2023-07-08T12:38:08.561730+0000","last_active":"2023-07-08T12:38:13.606000+0000","last_peered":"2023-07-08T12:38:13.606000+0000","last_clean":"2023-07-08T12:38:13.606000+0000","last_became_active":"2023-07-08T12:38:08.561498+0000","last_became_peered":"2023-07-08T12:38:08.561498+0000","last_unstale":"2023-07-08T12:38:13.606000+0000","last_undegraded":"2023-07-08T12:38:13.606000+0000","last_fullsized":"2023-07-08T12:38:13.606000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:06:06.921530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759585+0000","last_change":"2023-07-08T12:38:08.568400+0000","last_active":"2023-07-08T12:38:13.759585+0000","last_peered":"2023-07-08T12:38:13.759585+0000","last_clean":"2023-07-08T12:38:13.759585+0000","last_became_active":"2023-07-08T12:38:08.568070+0000","last_became_peered":"2023-07-08T12:38:08.568070+0000","last_unstale":"2023-07-08T12:38:13.759585+0000","last_undegraded":"2023-07-08T12:38:13.759585+0000","last_fullsized":"2023-07-08T12:38:13.759585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:35.332610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606886+0000","last_change":"2023-07-08T12:38:08.549069+0000","last_active":"2023-07-08T12:38:13.606886+0000","last_peered":"2023-07-08T12:38:13.606886+0000","last_clean":"2023-07-08T12:38:13.606886+0000","last_became_active":"2023-07-08T12:38:08.548774+0000","last_became_peered":"2023-07-08T12:38:08.548774+0000","last_unstale":"2023-07-08T12:38:13.606886+0000","last_undegraded":"2023-07-08T12:38:13.606886+0000","last_fullsized":"2023-07-08T12:38:13.606886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:02:05.917920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604270+0000","last_change":"2023-07-08T12:38:08.553458+0000","last_active":"2023-07-08T12:38:13.604270+0000","last_peered":"2023-07-08T12:38:13.604270+0000","last_clean":"2023-07-08T12:38:13.604270+0000","last_became_active":"2023-07-08T12:38:08.553177+0000","last_became_peered":"2023-07-08T12:38:08.553177+0000","last_unstale":"2023-07-08T12:38:13.604270+0000","last_undegraded":"2023-07-08T12:38:13.604270+0000","last_fullsized":"2023-07-08T12:38:13.604270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:07:08.413767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606438+0000","last_change":"2023-07-08T12:38:08.546039+0000","last_active":"2023-07-08T12:38:13.606438+0000","last_peered":"2023-07-08T12:38:13.606438+0000","last_clean":"2023-07-08T12:38:13.606438+0000","last_became_active":"2023-07-08T12:38:08.545723+0000","last_became_peered":"2023-07-08T12:38:08.545723+0000","last_unstale":"2023-07-08T12:38:13.606438+0000","last_undegraded":"2023-07-08T12:38:13.606438+0000","last_fullsized":"2023-07-08T12:38:13.606438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:05:03.136361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758578+0000","last_change":"2023-07-08T12:38:08.558172+0000","last_active":"2023-07-08T12:38:13.758578+0000","last_peered":"2023-07-08T12:38:13.758578+0000","last_clean":"2023-07-08T12:38:13.758578+0000","last_became_active":"2023-07-08T12:38:08.557888+0000","last_became_peered":"2023-07-08T12:38:08.557888+0000","last_unstale":"2023-07-08T12:38:13.758578+0000","last_undegraded":"2023-07-08T12:38:13.758578+0000","last_fullsized":"2023-07-08T12:38:13.758578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:37.290100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758835+0000","last_change":"2023-07-08T12:38:08.569776+0000","last_active":"2023-07-08T12:38:13.758835+0000","last_peered":"2023-07-08T12:38:13.758835+0000","last_clean":"2023-07-08T12:38:13.758835+0000","last_became_active":"2023-07-08T12:38:08.569530+0000","last_became_peered":"2023-07-08T12:38:08.569530+0000","last_unstale":"2023-07-08T12:38:13.758835+0000","last_undegraded":"2023-07-08T12:38:13.758835+0000","last_fullsized":"2023-07-08T12:38:13.758835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:18:27.333138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759672+0000","last_change":"2023-07-08T12:38:08.557720+0000","last_active":"2023-07-08T12:38:13.759672+0000","last_peered":"2023-07-08T12:38:13.759672+0000","last_clean":"2023-07-08T12:38:13.759672+0000","last_became_active":"2023-07-08T12:38:08.557390+0000","last_became_peered":"2023-07-08T12:38:08.557390+0000","last_unstale":"2023-07-08T12:38:13.759672+0000","last_undegraded":"2023-07-08T12:38:13.759672+0000","last_fullsized":"2023-07-08T12:38:13.759672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:13:13.582721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758123+0000","last_change":"2023-07-08T12:38:08.558048+0000","last_active":"2023-07-08T12:38:13.758123+0000","last_peered":"2023-07-08T12:38:13.758123+0000","last_clean":"2023-07-08T12:38:13.758123+0000","last_became_active":"2023-07-08T12:38:08.557871+0000","last_became_peered":"2023-07-08T12:38:08.557871+0000","last_unstale":"2023-07-08T12:38:13.758123+0000","last_undegraded":"2023-07-08T12:38:13.758123+0000","last_fullsized":"2023-07-08T12:38:13.758123+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:40.634514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759316+0000","last_change":"2023-07-08T12:38:08.553223+0000","last_active":"2023-07-08T12:38:13.759316+0000","last_peered":"2023-07-08T12:38:13.759316+0000","last_clean":"2023-07-08T12:38:13.759316+0000","last_became_active":"2023-07-08T12:38:08.552416+0000","last_became_peered":"2023-07-08T12:38:08.552416+0000","last_unstale":"2023-07-08T12:38:13.759316+0000","last_undegraded":"2023-07-08T12:38:13.759316+0000","last_fullsized":"2023-07-08T12:38:13.759316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:06:26.551781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760263+0000","last_change":"2023-07-08T12:38:08.553769+0000","last_active":"2023-07-08T12:38:13.760263+0000","last_peered":"2023-07-08T12:38:13.760263+0000","last_clean":"2023-07-08T12:38:13.760263+0000","last_became_active":"2023-07-08T12:38:08.553502+0000","last_became_peered":"2023-07-08T12:38:08.553502+0000","last_unstale":"2023-07-08T12:38:13.760263+0000","last_undegraded":"2023-07-08T12:38:13.760263+0000","last_fullsized":"2023-07-08T12:38:13.760263+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:35.758126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605466+0000","last_change":"2023-07-08T12:38:08.546308+0000","last_active":"2023-07-08T12:38:13.605466+0000","last_peered":"2023-07-08T12:38:13.605466+0000","last_clean":"2023-07-08T12:38:13.605466+0000","last_became_active":"2023-07-08T12:38:08.545865+0000","last_became_peered":"2023-07-08T12:38:08.545865+0000","last_unstale":"2023-07-08T12:38:13.605466+0000","last_undegraded":"2023-07-08T12:38:13.605466+0000","last_fullsized":"2023-07-08T12:38:13.605466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:22.667819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606681+0000","last_change":"2023-07-08T12:38:12.584534+0000","last_active":"2023-07-08T12:38:13.606681+0000","last_peered":"2023-07-08T12:38:13.606681+0000","last_clean":"2023-07-08T12:38:13.606681+0000","last_became_active":"2023-07-08T12:38:12.584153+0000","last_became_peered":"2023-07-08T12:38:12.584153+0000","last_unstale":"2023-07-08T12:38:13.606681+0000","last_undegraded":"2023-07-08T12:38:13.606681+0000","last_fullsized":"2023-07-08T12:38:13.606681+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:21:34.996680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606263+0000","last_change":"2023-07-08T12:38:07.547338+0000","last_active":"2023-07-08T12:38:13.606263+0000","last_peered":"2023-07-08T12:38:13.606263+0000","last_clean":"2023-07-08T12:38:13.606263+0000","last_became_active":"2023-07-08T12:38:07.547161+0000","last_became_peered":"2023-07-08T12:38:07.547161+0000","last_unstale":"2023-07-08T12:38:13.606263+0000","last_undegraded":"2023-07-08T12:38:13.606263+0000","last_fullsized":"2023-07-08T12:38:13.606263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:37:52.614496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759146+0000","last_change":"2023-07-08T12:38:08.553623+0000","last_active":"2023-07-08T12:38:13.759146+0000","last_peered":"2023-07-08T12:38:13.759146+0000","last_clean":"2023-07-08T12:38:13.759146+0000","last_became_active":"2023-07-08T12:38:08.551973+0000","last_became_peered":"2023-07-08T12:38:08.551973+0000","last_unstale":"2023-07-08T12:38:13.759146+0000","last_undegraded":"2023-07-08T12:38:13.759146+0000","last_fullsized":"2023-07-08T12:38:13.759146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:55:50.492319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759324+0000","last_change":"2023-07-08T12:38:12.590297+0000","last_active":"2023-07-08T12:38:13.759324+0000","last_peered":"2023-07-08T12:38:13.759324+0000","last_clean":"2023-07-08T12:38:13.759324+0000","last_became_active":"2023-07-08T12:38:12.589898+0000","last_became_peered":"2023-07-08T12:38:12.589898+0000","last_unstale":"2023-07-08T12:38:13.759324+0000","last_undegraded":"2023-07-08T12:38:13.759324+0000","last_fullsized":"2023-07-08T12:38:13.759324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:55.150008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758590+0000","last_change":"2023-07-08T12:38:07.548760+0000","last_active":"2023-07-08T12:38:13.758590+0000","last_peered":"2023-07-08T12:38:13.758590+0000","last_clean":"2023-07-08T12:38:13.758590+0000","last_became_active":"2023-07-08T12:38:07.548448+0000","last_became_peered":"2023-07-08T12:38:07.548448+0000","last_unstale":"2023-07-08T12:38:13.758590+0000","last_undegraded":"2023-07-08T12:38:13.758590+0000","last_fullsized":"2023-07-08T12:38:13.758590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T16:04:24.797809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759344+0000","last_change":"2023-07-08T12:38:08.561715+0000","last_active":"2023-07-08T12:38:13.759344+0000","last_peered":"2023-07-08T12:38:13.759344+0000","last_clean":"2023-07-08T12:38:13.759344+0000","last_became_active":"2023-07-08T12:38:08.561004+0000","last_became_peered":"2023-07-08T12:38:08.561004+0000","last_unstale":"2023-07-08T12:38:13.759344+0000","last_undegraded":"2023-07-08T12:38:13.759344+0000","last_fullsized":"2023-07-08T12:38:13.759344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:40.634514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759948+0000","last_change":"2023-07-08T12:38:12.593094+0000","last_active":"2023-07-08T12:38:13.759948+0000","last_peered":"2023-07-08T12:38:13.759948+0000","last_clean":"2023-07-08T12:38:13.759948+0000","last_became_active":"2023-07-08T12:38:12.592704+0000","last_became_peered":"2023-07-08T12:38:12.592704+0000","last_unstale":"2023-07-08T12:38:13.759948+0000","last_undegraded":"2023-07-08T12:38:13.759948+0000","last_fullsized":"2023-07-08T12:38:13.759948+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:26.710918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758339+0000","last_change":"2023-07-08T12:38:07.543437+0000","last_active":"2023-07-08T12:38:13.758339+0000","last_peered":"2023-07-08T12:38:13.758339+0000","last_clean":"2023-07-08T12:38:13.758339+0000","last_became_active":"2023-07-08T12:38:07.543158+0000","last_became_peered":"2023-07-08T12:38:07.543158+0000","last_unstale":"2023-07-08T12:38:13.758339+0000","last_undegraded":"2023-07-08T12:38:13.758339+0000","last_fullsized":"2023-07-08T12:38:13.758339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:48.497459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759270+0000","last_change":"2023-07-08T12:38:08.552907+0000","last_active":"2023-07-08T12:38:13.759270+0000","last_peered":"2023-07-08T12:38:13.759270+0000","last_clean":"2023-07-08T12:38:13.759270+0000","last_became_active":"2023-07-08T12:38:08.552286+0000","last_became_peered":"2023-07-08T12:38:08.552286+0000","last_unstale":"2023-07-08T12:38:13.759270+0000","last_undegraded":"2023-07-08T12:38:13.759270+0000","last_fullsized":"2023-07-08T12:38:13.759270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:11:32.912205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758741+0000","last_change":"2023-07-08T12:38:12.590030+0000","last_active":"2023-07-08T12:38:13.758741+0000","last_peered":"2023-07-08T12:38:13.758741+0000","last_clean":"2023-07-08T12:38:13.758741+0000","last_became_active":"2023-07-08T12:38:12.589580+0000","last_became_peered":"2023-07-08T12:38:12.589580+0000","last_unstale":"2023-07-08T12:38:13.758741+0000","last_undegraded":"2023-07-08T12:38:13.758741+0000","last_fullsized":"2023-07-08T12:38:13.758741+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:08:35.080577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759616+0000","last_change":"2023-07-08T12:38:07.541564+0000","last_active":"2023-07-08T12:38:13.759616+0000","last_peered":"2023-07-08T12:38:13.759616+0000","last_clean":"2023-07-08T12:38:13.759616+0000","last_became_active":"2023-07-08T12:38:07.540886+0000","last_became_peered":"2023-07-08T12:38:07.540886+0000","last_unstale":"2023-07-08T12:38:13.759616+0000","last_undegraded":"2023-07-08T12:38:13.759616+0000","last_fullsized":"2023-07-08T12:38:13.759616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:35.414408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758276+0000","last_change":"2023-07-08T12:38:08.556742+0000","last_active":"2023-07-08T12:38:13.758276+0000","last_peered":"2023-07-08T12:38:13.758276+0000","last_clean":"2023-07-08T12:38:13.758276+0000","last_became_active":"2023-07-08T12:38:08.556264+0000","last_became_peered":"2023-07-08T12:38:08.556264+0000","last_unstale":"2023-07-08T12:38:13.758276+0000","last_undegraded":"2023-07-08T12:38:13.758276+0000","last_fullsized":"2023-07-08T12:38:13.758276+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:56.156240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758009+0000","last_change":"2023-07-08T12:38:12.587899+0000","last_active":"2023-07-08T12:38:13.758009+0000","last_peered":"2023-07-08T12:38:13.758009+0000","last_clean":"2023-07-08T12:38:13.758009+0000","last_became_active":"2023-07-08T12:38:12.587591+0000","last_became_peered":"2023-07-08T12:38:12.587591+0000","last_unstale":"2023-07-08T12:38:13.758009+0000","last_undegraded":"2023-07-08T12:38:13.758009+0000","last_fullsized":"2023-07-08T12:38:13.758009+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:19.333590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605565+0000","last_change":"2023-07-08T12:38:07.536496+0000","last_active":"2023-07-08T12:38:13.605565+0000","last_peered":"2023-07-08T12:38:13.605565+0000","last_clean":"2023-07-08T12:38:13.605565+0000","last_became_active":"2023-07-08T12:38:07.535995+0000","last_became_peered":"2023-07-08T12:38:07.535995+0000","last_unstale":"2023-07-08T12:38:13.605565+0000","last_undegraded":"2023-07-08T12:38:13.605565+0000","last_fullsized":"2023-07-08T12:38:13.605565+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:28:48.361016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758283+0000","last_change":"2023-07-08T12:38:08.553479+0000","last_active":"2023-07-08T12:38:13.758283+0000","last_peered":"2023-07-08T12:38:13.758283+0000","last_clean":"2023-07-08T12:38:13.758283+0000","last_became_active":"2023-07-08T12:38:08.550664+0000","last_became_peered":"2023-07-08T12:38:08.550664+0000","last_unstale":"2023-07-08T12:38:13.758283+0000","last_undegraded":"2023-07-08T12:38:13.758283+0000","last_fullsized":"2023-07-08T12:38:13.758283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758974+0000","last_change":"2023-07-08T12:38:12.592991+0000","last_active":"2023-07-08T12:38:13.758974+0000","last_peered":"2023-07-08T12:38:13.758974+0000","last_clean":"2023-07-08T12:38:13.758974+0000","last_became_active":"2023-07-08T12:38:12.591995+0000","last_became_peered":"2023-07-08T12:38:12.591995+0000","last_unstale":"2023-07-08T12:38:13.758974+0000","last_undegraded":"2023-07-08T12:38:13.758974+0000","last_fullsized":"2023-07-08T12:38:13.758974+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:07.433127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759097+0000","last_change":"2023-07-08T12:38:07.551354+0000","last_active":"2023-07-08T12:38:13.759097+0000","last_peered":"2023-07-08T12:38:13.759097+0000","last_clean":"2023-07-08T12:38:13.759097+0000","last_became_active":"2023-07-08T12:38:07.551134+0000","last_became_peered":"2023-07-08T12:38:07.551134+0000","last_unstale":"2023-07-08T12:38:13.759097+0000","last_undegraded":"2023-07-08T12:38:13.759097+0000","last_fullsized":"2023-07-08T12:38:13.759097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:41:53.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760978+0000","last_change":"2023-07-08T12:38:08.558470+0000","last_active":"2023-07-08T12:38:13.760978+0000","last_peered":"2023-07-08T12:38:13.760978+0000","last_clean":"2023-07-08T12:38:13.760978+0000","last_became_active":"2023-07-08T12:38:08.557894+0000","last_became_peered":"2023-07-08T12:38:08.557894+0000","last_unstale":"2023-07-08T12:38:13.760978+0000","last_undegraded":"2023-07-08T12:38:13.760978+0000","last_fullsized":"2023-07-08T12:38:13.760978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:35.332610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.757971+0000","last_change":"2023-07-08T12:38:12.590703+0000","last_active":"2023-07-08T12:38:13.757971+0000","last_peered":"2023-07-08T12:38:13.757971+0000","last_clean":"2023-07-08T12:38:13.757971+0000","last_became_active":"2023-07-08T12:38:12.590268+0000","last_became_peered":"2023-07-08T12:38:12.590268+0000","last_unstale":"2023-07-08T12:38:13.757971+0000","last_undegraded":"2023-07-08T12:38:13.757971+0000","last_fullsized":"2023-07-08T12:38:13.757971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:35:36.579637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:23.855571+0000","last_change":"2023-07-08T12:38:07.544325+0000","last_active":"2023-07-08T12:38:23.855571+0000","last_peered":"2023-07-08T12:38:23.855571+0000","last_clean":"2023-07-08T12:38:23.855571+0000","last_became_active":"2023-07-08T12:38:07.543861+0000","last_became_peered":"2023-07-08T12:38:07.543861+0000","last_unstale":"2023-07-08T12:38:23.855571+0000","last_undegraded":"2023-07-08T12:38:23.855571+0000","last_fullsized":"2023-07-08T12:38:23.855571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:40:52.689398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605326+0000","last_change":"2023-07-08T12:38:08.553055+0000","last_active":"2023-07-08T12:38:13.605326+0000","last_peered":"2023-07-08T12:38:13.605326+0000","last_clean":"2023-07-08T12:38:13.605326+0000","last_became_active":"2023-07-08T12:38:08.552632+0000","last_became_peered":"2023-07-08T12:38:08.552632+0000","last_unstale":"2023-07-08T12:38:13.605326+0000","last_undegraded":"2023-07-08T12:38:13.605326+0000","last_fullsized":"2023-07-08T12:38:13.605326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:37:47.885139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605700+0000","last_change":"2023-07-08T12:38:12.598190+0000","last_active":"2023-07-08T12:38:13.605700+0000","last_peered":"2023-07-08T12:38:13.605700+0000","last_clean":"2023-07-08T12:38:13.605700+0000","last_became_active":"2023-07-08T12:38:12.597813+0000","last_became_peered":"2023-07-08T12:38:12.597813+0000","last_unstale":"2023-07-08T12:38:13.605700+0000","last_undegraded":"2023-07-08T12:38:13.605700+0000","last_fullsized":"2023-07-08T12:38:13.605700+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:41:58.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758412+0000","last_change":"2023-07-08T12:38:07.543441+0000","last_active":"2023-07-08T12:38:13.758412+0000","last_peered":"2023-07-08T12:38:13.758412+0000","last_clean":"2023-07-08T12:38:13.758412+0000","last_became_active":"2023-07-08T12:38:07.543188+0000","last_became_peered":"2023-07-08T12:38:07.543188+0000","last_unstale":"2023-07-08T12:38:13.758412+0000","last_undegraded":"2023-07-08T12:38:13.758412+0000","last_fullsized":"2023-07-08T12:38:13.758412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:34.328010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758440+0000","last_change":"2023-07-08T12:38:08.552797+0000","last_active":"2023-07-08T12:38:13.758440+0000","last_peered":"2023-07-08T12:38:13.758440+0000","last_clean":"2023-07-08T12:38:13.758440+0000","last_became_active":"2023-07-08T12:38:08.552289+0000","last_became_peered":"2023-07-08T12:38:08.552289+0000","last_unstale":"2023-07-08T12:38:13.758440+0000","last_undegraded":"2023-07-08T12:38:13.758440+0000","last_fullsized":"2023-07-08T12:38:13.758440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:07.081181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758345+0000","last_change":"2023-07-08T12:38:12.592793+0000","last_active":"2023-07-08T12:38:13.758345+0000","last_peered":"2023-07-08T12:38:13.758345+0000","last_clean":"2023-07-08T12:38:13.758345+0000","last_became_active":"2023-07-08T12:38:12.592414+0000","last_became_peered":"2023-07-08T12:38:12.592414+0000","last_unstale":"2023-07-08T12:38:13.758345+0000","last_undegraded":"2023-07-08T12:38:13.758345+0000","last_fullsized":"2023-07-08T12:38:13.758345+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:39.801225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758616+0000","last_change":"2023-07-08T12:38:07.538612+0000","last_active":"2023-07-08T12:38:13.758616+0000","last_peered":"2023-07-08T12:38:13.758616+0000","last_clean":"2023-07-08T12:38:13.758616+0000","last_became_active":"2023-07-08T12:38:07.538048+0000","last_became_peered":"2023-07-08T12:38:07.538048+0000","last_unstale":"2023-07-08T12:38:13.758616+0000","last_undegraded":"2023-07-08T12:38:13.758616+0000","last_fullsized":"2023-07-08T12:38:13.758616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:04:41.955662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605266+0000","last_change":"2023-07-08T12:38:08.546131+0000","last_active":"2023-07-08T12:38:13.605266+0000","last_peered":"2023-07-08T12:38:13.605266+0000","last_clean":"2023-07-08T12:38:13.605266+0000","last_became_active":"2023-07-08T12:38:08.545491+0000","last_became_peered":"2023-07-08T12:38:08.545491+0000","last_unstale":"2023-07-08T12:38:13.605266+0000","last_undegraded":"2023-07-08T12:38:13.605266+0000","last_fullsized":"2023-07-08T12:38:13.605266+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:36.419008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758428+0000","last_change":"2023-07-08T12:38:12.584683+0000","last_active":"2023-07-08T12:38:13.758428+0000","last_peered":"2023-07-08T12:38:13.758428+0000","last_clean":"2023-07-08T12:38:13.758428+0000","last_became_active":"2023-07-08T12:38:12.584256+0000","last_became_peered":"2023-07-08T12:38:12.584256+0000","last_unstale":"2023-07-08T12:38:13.758428+0000","last_undegraded":"2023-07-08T12:38:13.758428+0000","last_fullsized":"2023-07-08T12:38:13.758428+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:41.333199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758406+0000","last_change":"2023-07-08T12:38:07.543388+0000","last_active":"2023-07-08T12:38:13.758406+0000","last_peered":"2023-07-08T12:38:13.758406+0000","last_clean":"2023-07-08T12:38:13.758406+0000","last_became_active":"2023-07-08T12:38:07.543117+0000","last_became_peered":"2023-07-08T12:38:07.543117+0000","last_unstale":"2023-07-08T12:38:13.758406+0000","last_undegraded":"2023-07-08T12:38:13.758406+0000","last_fullsized":"2023-07-08T12:38:13.758406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:21.663219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759864+0000","last_change":"2023-07-08T12:38:08.561895+0000","last_active":"2023-07-08T12:38:13.759864+0000","last_peered":"2023-07-08T12:38:13.759864+0000","last_clean":"2023-07-08T12:38:13.759864+0000","last_became_active":"2023-07-08T12:38:08.561428+0000","last_became_peered":"2023-07-08T12:38:08.561428+0000","last_unstale":"2023-07-08T12:38:13.759864+0000","last_undegraded":"2023-07-08T12:38:13.759864+0000","last_fullsized":"2023-07-08T12:38:13.759864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:07.081181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605310+0000","last_change":"2023-07-08T12:38:12.592292+0000","last_active":"2023-07-08T12:38:13.605310+0000","last_peered":"2023-07-08T12:38:13.605310+0000","last_clean":"2023-07-08T12:38:13.605310+0000","last_became_active":"2023-07-08T12:38:12.591964+0000","last_became_peered":"2023-07-08T12:38:12.591964+0000","last_unstale":"2023-07-08T12:38:13.605310+0000","last_undegraded":"2023-07-08T12:38:13.605310+0000","last_fullsized":"2023-07-08T12:38:13.605310+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:53.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758158+0000","last_change":"2023-07-08T12:38:07.543683+0000","last_active":"2023-07-08T12:38:13.758158+0000","last_peered":"2023-07-08T12:38:13.758158+0000","last_clean":"2023-07-08T12:38:13.758158+0000","last_became_active":"2023-07-08T12:38:07.543416+0000","last_became_peered":"2023-07-08T12:38:07.543416+0000","last_unstale":"2023-07-08T12:38:13.758158+0000","last_undegraded":"2023-07-08T12:38:13.758158+0000","last_fullsized":"2023-07-08T12:38:13.758158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:56:16.609129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758366+0000","last_change":"2023-07-08T12:38:08.556977+0000","last_active":"2023-07-08T12:38:13.758366+0000","last_peered":"2023-07-08T12:38:13.758366+0000","last_clean":"2023-07-08T12:38:13.758366+0000","last_became_active":"2023-07-08T12:38:08.555976+0000","last_became_peered":"2023-07-08T12:38:08.555976+0000","last_unstale":"2023-07-08T12:38:13.758366+0000","last_undegraded":"2023-07-08T12:38:13.758366+0000","last_fullsized":"2023-07-08T12:38:13.758366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:36.419008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759039+0000","last_change":"2023-07-08T12:38:12.582526+0000","last_active":"2023-07-08T12:38:13.759039+0000","last_peered":"2023-07-08T12:38:13.759039+0000","last_clean":"2023-07-08T12:38:13.759039+0000","last_became_active":"2023-07-08T12:38:12.582130+0000","last_became_peered":"2023-07-08T12:38:12.582130+0000","last_unstale":"2023-07-08T12:38:13.759039+0000","last_undegraded":"2023-07-08T12:38:13.759039+0000","last_fullsized":"2023-07-08T12:38:13.759039+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:31:58.033845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759843+0000","last_change":"2023-07-08T12:38:07.543929+0000","last_active":"2023-07-08T12:38:13.759843+0000","last_peered":"2023-07-08T12:38:13.759843+0000","last_clean":"2023-07-08T12:38:13.759843+0000","last_became_active":"2023-07-08T12:38:07.543693+0000","last_became_peered":"2023-07-08T12:38:07.543693+0000","last_unstale":"2023-07-08T12:38:13.759843+0000","last_undegraded":"2023-07-08T12:38:13.759843+0000","last_fullsized":"2023-07-08T12:38:13.759843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:55:49.487719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604436+0000","last_change":"2023-07-08T12:38:08.548856+0000","last_active":"2023-07-08T12:38:13.604436+0000","last_peered":"2023-07-08T12:38:13.604436+0000","last_clean":"2023-07-08T12:38:13.604436+0000","last_became_active":"2023-07-08T12:38:08.548294+0000","last_became_peered":"2023-07-08T12:38:08.548294+0000","last_unstale":"2023-07-08T12:38:13.604436+0000","last_undegraded":"2023-07-08T12:38:13.604436+0000","last_fullsized":"2023-07-08T12:38:13.604436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:53.873880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.603862+0000","last_change":"2023-07-08T12:38:12.587147+0000","last_active":"2023-07-08T12:38:13.603862+0000","last_peered":"2023-07-08T12:38:13.603862+0000","last_clean":"2023-07-08T12:38:13.603862+0000","last_became_active":"2023-07-08T12:38:12.586709+0000","last_became_peered":"2023-07-08T12:38:12.586709+0000","last_unstale":"2023-07-08T12:38:13.603862+0000","last_undegraded":"2023-07-08T12:38:13.603862+0000","last_fullsized":"2023-07-08T12:38:13.603862+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T12:58:56.632605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759614+0000","last_change":"2023-07-08T12:38:07.545521+0000","last_active":"2023-07-08T12:38:13.759614+0000","last_peered":"2023-07-08T12:38:13.759614+0000","last_clean":"2023-07-08T12:38:13.759614+0000","last_became_active":"2023-07-08T12:38:07.545260+0000","last_became_peered":"2023-07-08T12:38:07.545260+0000","last_unstale":"2023-07-08T12:38:13.759614+0000","last_undegraded":"2023-07-08T12:38:13.759614+0000","last_fullsized":"2023-07-08T12:38:13.759614+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:02.385428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605419+0000","last_change":"2023-07-08T12:38:08.552222+0000","last_active":"2023-07-08T12:38:13.605419+0000","last_peered":"2023-07-08T12:38:13.605419+0000","last_clean":"2023-07-08T12:38:13.605419+0000","last_became_active":"2023-07-08T12:38:08.551940+0000","last_became_peered":"2023-07-08T12:38:08.551940+0000","last_unstale":"2023-07-08T12:38:13.605419+0000","last_undegraded":"2023-07-08T12:38:13.605419+0000","last_fullsized":"2023-07-08T12:38:13.605419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:31:34.141462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604126+0000","last_change":"2023-07-08T12:38:12.589202+0000","last_active":"2023-07-08T12:38:13.604126+0000","last_peered":"2023-07-08T12:38:13.604126+0000","last_clean":"2023-07-08T12:38:13.604126+0000","last_became_active":"2023-07-08T12:38:12.586775+0000","last_became_peered":"2023-07-08T12:38:12.586775+0000","last_unstale":"2023-07-08T12:38:13.604126+0000","last_undegraded":"2023-07-08T12:38:13.604126+0000","last_fullsized":"2023-07-08T12:38:13.604126+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:17:35.606731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758901+0000","last_change":"2023-07-08T12:38:07.542576+0000","last_active":"2023-07-08T12:38:13.758901+0000","last_peered":"2023-07-08T12:38:13.758901+0000","last_clean":"2023-07-08T12:38:13.758901+0000","last_became_active":"2023-07-08T12:38:07.542296+0000","last_became_peered":"2023-07-08T12:38:07.542296+0000","last_unstale":"2023-07-08T12:38:13.758901+0000","last_undegraded":"2023-07-08T12:38:13.758901+0000","last_fullsized":"2023-07-08T12:38:13.758901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:28:48.361016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758718+0000","last_change":"2023-07-08T12:38:08.560794+0000","last_active":"2023-07-08T12:38:13.758718+0000","last_peered":"2023-07-08T12:38:13.758718+0000","last_clean":"2023-07-08T12:38:13.758718+0000","last_became_active":"2023-07-08T12:38:08.560405+0000","last_became_peered":"2023-07-08T12:38:08.560405+0000","last_unstale":"2023-07-08T12:38:13.758718+0000","last_undegraded":"2023-07-08T12:38:13.758718+0000","last_fullsized":"2023-07-08T12:38:13.758718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:52:43.348476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604255+0000","last_change":"2023-07-08T12:38:12.585051+0000","last_active":"2023-07-08T12:38:13.604255+0000","last_peered":"2023-07-08T12:38:13.604255+0000","last_clean":"2023-07-08T12:38:13.604255+0000","last_became_active":"2023-07-08T12:38:12.583082+0000","last_became_peered":"2023-07-08T12:38:12.583082+0000","last_unstale":"2023-07-08T12:38:13.604255+0000","last_undegraded":"2023-07-08T12:38:13.604255+0000","last_fullsized":"2023-07-08T12:38:13.604255+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:02:43.298466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604866+0000","last_change":"2023-07-08T12:38:07.537900+0000","last_active":"2023-07-08T12:38:13.604866+0000","last_peered":"2023-07-08T12:38:13.604866+0000","last_clean":"2023-07-08T12:38:13.604866+0000","last_became_active":"2023-07-08T12:38:07.536896+0000","last_became_peered":"2023-07-08T12:38:07.536896+0000","last_unstale":"2023-07-08T12:38:13.604866+0000","last_undegraded":"2023-07-08T12:38:13.604866+0000","last_fullsized":"2023-07-08T12:38:13.604866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:55.151640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759269+0000","last_change":"2023-07-08T12:38:08.569854+0000","last_active":"2023-07-08T12:38:13.759269+0000","last_peered":"2023-07-08T12:38:13.759269+0000","last_clean":"2023-07-08T12:38:13.759269+0000","last_became_active":"2023-07-08T12:38:08.569549+0000","last_became_peered":"2023-07-08T12:38:08.569549+0000","last_unstale":"2023-07-08T12:38:13.759269+0000","last_undegraded":"2023-07-08T12:38:13.759269+0000","last_fullsized":"2023-07-08T12:38:13.759269+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:15.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604571+0000","last_change":"2023-07-08T12:38:12.580215+0000","last_active":"2023-07-08T12:38:13.604571+0000","last_peered":"2023-07-08T12:38:13.604571+0000","last_clean":"2023-07-08T12:38:13.604571+0000","last_became_active":"2023-07-08T12:38:12.579706+0000","last_became_peered":"2023-07-08T12:38:12.579706+0000","last_unstale":"2023-07-08T12:38:13.604571+0000","last_undegraded":"2023-07-08T12:38:13.604571+0000","last_fullsized":"2023-07-08T12:38:13.604571+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:08:35.080577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758601+0000","last_change":"2023-07-08T12:38:07.542376+0000","last_active":"2023-07-08T12:38:13.758601+0000","last_peered":"2023-07-08T12:38:13.758601+0000","last_clean":"2023-07-08T12:38:13.758601+0000","last_became_active":"2023-07-08T12:38:07.542046+0000","last_became_peered":"2023-07-08T12:38:07.542046+0000","last_unstale":"2023-07-08T12:38:13.758601+0000","last_undegraded":"2023-07-08T12:38:13.758601+0000","last_fullsized":"2023-07-08T12:38:13.758601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:28:48.361016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605011+0000","last_change":"2023-07-08T12:38:08.553132+0000","last_active":"2023-07-08T12:38:13.605011+0000","last_peered":"2023-07-08T12:38:13.605011+0000","last_clean":"2023-07-08T12:38:13.605011+0000","last_became_active":"2023-07-08T12:38:08.552711+0000","last_became_peered":"2023-07-08T12:38:08.552711+0000","last_unstale":"2023-07-08T12:38:13.605011+0000","last_undegraded":"2023-07-08T12:38:13.605011+0000","last_fullsized":"2023-07-08T12:38:13.605011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T16:24:56.036654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759123+0000","last_change":"2023-07-08T12:38:12.597633+0000","last_active":"2023-07-08T12:38:13.759123+0000","last_peered":"2023-07-08T12:38:13.759123+0000","last_clean":"2023-07-08T12:38:13.759123+0000","last_became_active":"2023-07-08T12:38:12.597211+0000","last_became_peered":"2023-07-08T12:38:12.597211+0000","last_unstale":"2023-07-08T12:38:13.759123+0000","last_undegraded":"2023-07-08T12:38:13.759123+0000","last_fullsized":"2023-07-08T12:38:13.759123+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:04:47.003361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758887+0000","last_change":"2023-07-08T12:38:07.542040+0000","last_active":"2023-07-08T12:38:13.758887+0000","last_peered":"2023-07-08T12:38:13.758887+0000","last_clean":"2023-07-08T12:38:13.758887+0000","last_became_active":"2023-07-08T12:38:07.541743+0000","last_became_peered":"2023-07-08T12:38:07.541743+0000","last_unstale":"2023-07-08T12:38:13.758887+0000","last_undegraded":"2023-07-08T12:38:13.758887+0000","last_fullsized":"2023-07-08T12:38:13.758887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:41:49.220614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606520+0000","last_change":"2023-07-08T12:38:08.548025+0000","last_active":"2023-07-08T12:38:13.606520+0000","last_peered":"2023-07-08T12:38:13.606520+0000","last_clean":"2023-07-08T12:38:13.606520+0000","last_became_active":"2023-07-08T12:38:08.547723+0000","last_became_peered":"2023-07-08T12:38:08.547723+0000","last_unstale":"2023-07-08T12:38:13.606520+0000","last_undegraded":"2023-07-08T12:38:13.606520+0000","last_fullsized":"2023-07-08T12:38:13.606520+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:01:21.861852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605658+0000","last_change":"2023-07-08T12:38:12.587540+0000","last_active":"2023-07-08T12:38:13.605658+0000","last_peered":"2023-07-08T12:38:13.605658+0000","last_clean":"2023-07-08T12:38:13.605658+0000","last_became_active":"2023-07-08T12:38:12.587180+0000","last_became_peered":"2023-07-08T12:38:12.587180+0000","last_unstale":"2023-07-08T12:38:13.605658+0000","last_undegraded":"2023-07-08T12:38:13.605658+0000","last_fullsized":"2023-07-08T12:38:13.605658+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:31:58.033845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605007+0000","last_change":"2023-07-08T12:38:07.535635+0000","last_active":"2023-07-08T12:38:13.605007+0000","last_peered":"2023-07-08T12:38:13.605007+0000","last_clean":"2023-07-08T12:38:13.605007+0000","last_became_active":"2023-07-08T12:38:07.535335+0000","last_became_peered":"2023-07-08T12:38:07.535335+0000","last_unstale":"2023-07-08T12:38:13.605007+0000","last_undegraded":"2023-07-08T12:38:13.605007+0000","last_fullsized":"2023-07-08T12:38:13.605007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:15:42.329406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606965+0000","last_change":"2023-07-08T12:38:08.553024+0000","last_active":"2023-07-08T12:38:13.606965+0000","last_peered":"2023-07-08T12:38:13.606965+0000","last_clean":"2023-07-08T12:38:13.606965+0000","last_became_active":"2023-07-08T12:38:08.552696+0000","last_became_peered":"2023-07-08T12:38:08.552696+0000","last_unstale":"2023-07-08T12:38:13.606965+0000","last_undegraded":"2023-07-08T12:38:13.606965+0000","last_fullsized":"2023-07-08T12:38:13.606965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:33:05.815083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606715+0000","last_change":"2023-07-08T12:38:12.584493+0000","last_active":"2023-07-08T12:38:13.606715+0000","last_peered":"2023-07-08T12:38:13.606715+0000","last_clean":"2023-07-08T12:38:13.606715+0000","last_became_active":"2023-07-08T12:38:12.584087+0000","last_became_peered":"2023-07-08T12:38:12.584087+0000","last_unstale":"2023-07-08T12:38:13.606715+0000","last_undegraded":"2023-07-08T12:38:13.606715+0000","last_fullsized":"2023-07-08T12:38:13.606715+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:10:54.808570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758207+0000","last_change":"2023-07-08T12:38:07.540092+0000","last_active":"2023-07-08T12:38:13.758207+0000","last_peered":"2023-07-08T12:38:13.758207+0000","last_clean":"2023-07-08T12:38:13.758207+0000","last_became_active":"2023-07-08T12:38:07.539789+0000","last_became_peered":"2023-07-08T12:38:07.539789+0000","last_unstale":"2023-07-08T12:38:13.758207+0000","last_undegraded":"2023-07-08T12:38:13.758207+0000","last_fullsized":"2023-07-08T12:38:13.758207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:39.629914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759601+0000","last_change":"2023-07-08T12:38:08.564633+0000","last_active":"2023-07-08T12:38:13.759601+0000","last_peered":"2023-07-08T12:38:13.759601+0000","last_clean":"2023-07-08T12:38:13.759601+0000","last_became_active":"2023-07-08T12:38:08.564323+0000","last_became_peered":"2023-07-08T12:38:08.564323+0000","last_unstale":"2023-07-08T12:38:13.759601+0000","last_undegraded":"2023-07-08T12:38:13.759601+0000","last_fullsized":"2023-07-08T12:38:13.759601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:28.004982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759180+0000","last_change":"2023-07-08T12:38:12.586250+0000","last_active":"2023-07-08T12:38:13.759180+0000","last_peered":"2023-07-08T12:38:13.759180+0000","last_clean":"2023-07-08T12:38:13.759180+0000","last_became_active":"2023-07-08T12:38:12.585838+0000","last_became_peered":"2023-07-08T12:38:12.585838+0000","last_unstale":"2023-07-08T12:38:13.759180+0000","last_undegraded":"2023-07-08T12:38:13.759180+0000","last_fullsized":"2023-07-08T12:38:13.759180+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:06:30.594880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759317+0000","last_change":"2023-07-08T12:38:03.160177+0000","last_active":"2023-07-08T12:38:13.759317+0000","last_peered":"2023-07-08T12:38:13.759317+0000","last_clean":"2023-07-08T12:38:13.759317+0000","last_became_active":"2023-07-08T12:38:03.159864+0000","last_became_peered":"2023-07-08T12:38:03.159864+0000","last_unstale":"2023-07-08T12:38:13.759317+0000","last_undegraded":"2023-07-08T12:38:13.759317+0000","last_fullsized":"2023-07-08T12:38:13.759317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:31:56.449203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604812+0000","last_change":"2023-07-08T12:38:07.537328+0000","last_active":"2023-07-08T12:38:13.604812+0000","last_peered":"2023-07-08T12:38:13.604812+0000","last_clean":"2023-07-08T12:38:13.604812+0000","last_became_active":"2023-07-08T12:38:07.537038+0000","last_became_peered":"2023-07-08T12:38:07.537038+0000","last_unstale":"2023-07-08T12:38:13.604812+0000","last_undegraded":"2023-07-08T12:38:13.604812+0000","last_fullsized":"2023-07-08T12:38:13.604812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:39.629914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760774+0000","last_change":"2023-07-08T12:38:08.558271+0000","last_active":"2023-07-08T12:38:13.760774+0000","last_peered":"2023-07-08T12:38:13.760774+0000","last_clean":"2023-07-08T12:38:13.760774+0000","last_became_active":"2023-07-08T12:38:08.557598+0000","last_became_peered":"2023-07-08T12:38:08.557598+0000","last_unstale":"2023-07-08T12:38:13.760774+0000","last_undegraded":"2023-07-08T12:38:13.760774+0000","last_fullsized":"2023-07-08T12:38:13.760774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:49.502059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759252+0000","last_change":"2023-07-08T12:38:12.591995+0000","last_active":"2023-07-08T12:38:13.759252+0000","last_peered":"2023-07-08T12:38:13.759252+0000","last_clean":"2023-07-08T12:38:13.759252+0000","last_became_active":"2023-07-08T12:38:12.591624+0000","last_became_peered":"2023-07-08T12:38:12.591624+0000","last_unstale":"2023-07-08T12:38:13.759252+0000","last_undegraded":"2023-07-08T12:38:13.759252+0000","last_fullsized":"2023-07-08T12:38:13.759252+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:57.916979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605001+0000","last_change":"2023-07-08T12:38:02.384639+0000","last_active":"2023-07-08T12:38:13.605001+0000","last_peered":"2023-07-08T12:38:13.605001+0000","last_clean":"2023-07-08T12:38:13.605001+0000","last_became_active":"2023-07-08T12:38:02.384372+0000","last_became_peered":"2023-07-08T12:38:02.384372+0000","last_unstale":"2023-07-08T12:38:13.605001+0000","last_undegraded":"2023-07-08T12:38:13.605001+0000","last_fullsized":"2023-07-08T12:38:13.605001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:29.599162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605155+0000","last_change":"2023-07-08T12:38:07.541393+0000","last_active":"2023-07-08T12:38:13.605155+0000","last_peered":"2023-07-08T12:38:13.605155+0000","last_clean":"2023-07-08T12:38:13.605155+0000","last_became_active":"2023-07-08T12:38:07.541155+0000","last_became_peered":"2023-07-08T12:38:07.541155+0000","last_unstale":"2023-07-08T12:38:13.605155+0000","last_undegraded":"2023-07-08T12:38:13.605155+0000","last_fullsized":"2023-07-08T12:38:13.605155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:51:13.319094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759210+0000","last_change":"2023-07-08T12:38:08.550498+0000","last_active":"2023-07-08T12:38:13.759210+0000","last_peered":"2023-07-08T12:38:13.759210+0000","last_clean":"2023-07-08T12:38:13.759210+0000","last_became_active":"2023-07-08T12:38:08.550106+0000","last_became_peered":"2023-07-08T12:38:08.550106+0000","last_unstale":"2023-07-08T12:38:13.759210+0000","last_undegraded":"2023-07-08T12:38:13.759210+0000","last_fullsized":"2023-07-08T12:38:13.759210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:18:27.333138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604527+0000","last_change":"2023-07-08T12:38:12.586451+0000","last_active":"2023-07-08T12:38:13.604527+0000","last_peered":"2023-07-08T12:38:13.604527+0000","last_clean":"2023-07-08T12:38:13.604527+0000","last_became_active":"2023-07-08T12:38:12.582823+0000","last_became_peered":"2023-07-08T12:38:12.582823+0000","last_unstale":"2023-07-08T12:38:13.604527+0000","last_undegraded":"2023-07-08T12:38:13.604527+0000","last_fullsized":"2023-07-08T12:38:13.604527+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:18:34.491455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604236+0000","last_change":"2023-07-08T12:38:03.212466+0000","last_active":"2023-07-08T12:38:13.604236+0000","last_peered":"2023-07-08T12:38:13.604236+0000","last_clean":"2023-07-08T12:38:13.604236+0000","last_became_active":"2023-07-08T12:38:03.212182+0000","last_became_peered":"2023-07-08T12:38:03.212182+0000","last_unstale":"2023-07-08T12:38:13.604236+0000","last_undegraded":"2023-07-08T12:38:13.604236+0000","last_fullsized":"2023-07-08T12:38:13.604236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:06:34.067771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758091+0000","last_change":"2023-07-08T12:38:07.538415+0000","last_active":"2023-07-08T12:38:13.758091+0000","last_peered":"2023-07-08T12:38:13.758091+0000","last_clean":"2023-07-08T12:38:13.758091+0000","last_became_active":"2023-07-08T12:38:07.537871+0000","last_became_peered":"2023-07-08T12:38:07.537871+0000","last_unstale":"2023-07-08T12:38:13.758091+0000","last_undegraded":"2023-07-08T12:38:13.758091+0000","last_fullsized":"2023-07-08T12:38:13.758091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:00:59.245798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759478+0000","last_change":"2023-07-08T12:38:08.554865+0000","last_active":"2023-07-08T12:38:13.759478+0000","last_peered":"2023-07-08T12:38:13.759478+0000","last_clean":"2023-07-08T12:38:13.759478+0000","last_became_active":"2023-07-08T12:38:08.554591+0000","last_became_peered":"2023-07-08T12:38:08.554591+0000","last_unstale":"2023-07-08T12:38:13.759478+0000","last_undegraded":"2023-07-08T12:38:13.759478+0000","last_fullsized":"2023-07-08T12:38:13.759478+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:53.873880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758470+0000","last_change":"2023-07-08T12:38:12.579919+0000","last_active":"2023-07-08T12:38:13.758470+0000","last_peered":"2023-07-08T12:38:13.758470+0000","last_clean":"2023-07-08T12:38:13.758470+0000","last_became_active":"2023-07-08T12:38:12.579500+0000","last_became_peered":"2023-07-08T12:38:12.579500+0000","last_unstale":"2023-07-08T12:38:13.758470+0000","last_undegraded":"2023-07-08T12:38:13.758470+0000","last_fullsized":"2023-07-08T12:38:13.758470+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:53.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605571+0000","last_change":"2023-07-08T12:38:02.380175+0000","last_active":"2023-07-08T12:38:13.605571+0000","last_peered":"2023-07-08T12:38:13.605571+0000","last_clean":"2023-07-08T12:38:13.605571+0000","last_became_active":"2023-07-08T12:38:02.379867+0000","last_became_peered":"2023-07-08T12:38:02.379867+0000","last_unstale":"2023-07-08T12:38:13.605571+0000","last_undegraded":"2023-07-08T12:38:13.605571+0000","last_fullsized":"2023-07-08T12:38:13.605571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:35:57.492852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758117+0000","last_change":"2023-07-08T12:37:58.222371+0000","last_active":"2023-07-08T12:38:13.758117+0000","last_peered":"2023-07-08T12:38:13.758117+0000","last_clean":"2023-07-08T12:38:13.758117+0000","last_became_active":"2023-07-08T12:37:58.221381+0000","last_became_peered":"2023-07-08T12:37:58.221381+0000","last_unstale":"2023-07-08T12:38:13.758117+0000","last_undegraded":"2023-07-08T12:38:13.758117+0000","last_fullsized":"2023-07-08T12:38:13.758117+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:37:56.327863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:37:56.327863+0000","last_clean_scrub_stamp":"2023-07-08T12:37:56.327863+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:44.968733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758891+0000","last_change":"2023-07-08T12:38:07.536547+0000","last_active":"2023-07-08T12:38:13.758891+0000","last_peered":"2023-07-08T12:38:13.758891+0000","last_clean":"2023-07-08T12:38:13.758891+0000","last_became_active":"2023-07-08T12:38:07.536255+0000","last_became_peered":"2023-07-08T12:38:07.536255+0000","last_unstale":"2023-07-08T12:38:13.758891+0000","last_undegraded":"2023-07-08T12:38:13.758891+0000","last_fullsized":"2023-07-08T12:38:13.758891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:41:53.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759285+0000","last_change":"2023-07-08T12:38:08.550673+0000","last_active":"2023-07-08T12:38:13.759285+0000","last_peered":"2023-07-08T12:38:13.759285+0000","last_clean":"2023-07-08T12:38:13.759285+0000","last_became_active":"2023-07-08T12:38:08.550347+0000","last_became_peered":"2023-07-08T12:38:08.550347+0000","last_unstale":"2023-07-08T12:38:13.759285+0000","last_undegraded":"2023-07-08T12:38:13.759285+0000","last_fullsized":"2023-07-08T12:38:13.759285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:03:34.056111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605442+0000","last_change":"2023-07-08T12:38:12.597251+0000","last_active":"2023-07-08T12:38:13.605442+0000","last_peered":"2023-07-08T12:38:13.605442+0000","last_clean":"2023-07-08T12:38:13.605442+0000","last_became_active":"2023-07-08T12:38:12.596856+0000","last_became_peered":"2023-07-08T12:38:12.596856+0000","last_unstale":"2023-07-08T12:38:13.605442+0000","last_undegraded":"2023-07-08T12:38:13.605442+0000","last_fullsized":"2023-07-08T12:38:13.605442+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:11.124280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758299+0000","last_change":"2023-07-08T12:38:02.385193+0000","last_active":"2023-07-08T12:38:13.758299+0000","last_peered":"2023-07-08T12:38:13.758299+0000","last_clean":"2023-07-08T12:38:13.758299+0000","last_became_active":"2023-07-08T12:38:02.384865+0000","last_became_peered":"2023-07-08T12:38:02.384865+0000","last_unstale":"2023-07-08T12:38:13.758299+0000","last_undegraded":"2023-07-08T12:38:13.758299+0000","last_fullsized":"2023-07-08T12:38:13.758299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:30.260044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.757707+0000","last_change":"2023-07-08T12:38:07.543452+0000","last_active":"2023-07-08T12:38:13.757707+0000","last_peered":"2023-07-08T12:38:13.757707+0000","last_clean":"2023-07-08T12:38:13.757707+0000","last_became_active":"2023-07-08T12:38:07.543141+0000","last_became_peered":"2023-07-08T12:38:07.543141+0000","last_unstale":"2023-07-08T12:38:13.757707+0000","last_undegraded":"2023-07-08T12:38:13.757707+0000","last_fullsized":"2023-07-08T12:38:13.757707+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:03:16.870782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759081+0000","last_change":"2023-07-08T12:38:08.570468+0000","last_active":"2023-07-08T12:38:13.759081+0000","last_peered":"2023-07-08T12:38:13.759081+0000","last_clean":"2023-07-08T12:38:13.759081+0000","last_became_active":"2023-07-08T12:38:08.570183+0000","last_became_peered":"2023-07-08T12:38:08.570183+0000","last_unstale":"2023-07-08T12:38:13.759081+0000","last_undegraded":"2023-07-08T12:38:13.759081+0000","last_fullsized":"2023-07-08T12:38:13.759081+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:32:02.608167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759172+0000","last_change":"2023-07-08T12:38:12.597186+0000","last_active":"2023-07-08T12:38:13.759172+0000","last_peered":"2023-07-08T12:38:13.759172+0000","last_clean":"2023-07-08T12:38:13.759172+0000","last_became_active":"2023-07-08T12:38:12.596772+0000","last_became_peered":"2023-07-08T12:38:12.596772+0000","last_unstale":"2023-07-08T12:38:13.759172+0000","last_undegraded":"2023-07-08T12:38:13.759172+0000","last_fullsized":"2023-07-08T12:38:13.759172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:06:56.291254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758758+0000","last_change":"2023-07-08T12:38:03.160344+0000","last_active":"2023-07-08T12:38:13.758758+0000","last_peered":"2023-07-08T12:38:13.758758+0000","last_clean":"2023-07-08T12:38:13.758758+0000","last_became_active":"2023-07-08T12:38:03.160009+0000","last_became_peered":"2023-07-08T12:38:03.160009+0000","last_unstale":"2023-07-08T12:38:13.758758+0000","last_undegraded":"2023-07-08T12:38:13.758758+0000","last_fullsized":"2023-07-08T12:38:13.758758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:35:57.492852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759529+0000","last_change":"2023-07-08T12:38:07.539629+0000","last_active":"2023-07-08T12:38:13.759529+0000","last_peered":"2023-07-08T12:38:13.759529+0000","last_clean":"2023-07-08T12:38:13.759529+0000","last_became_active":"2023-07-08T12:38:07.537615+0000","last_became_peered":"2023-07-08T12:38:07.537615+0000","last_unstale":"2023-07-08T12:38:13.759529+0000","last_undegraded":"2023-07-08T12:38:13.759529+0000","last_fullsized":"2023-07-08T12:38:13.759529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:55.151640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759477+0000","last_change":"2023-07-08T12:38:08.556275+0000","last_active":"2023-07-08T12:38:13.759477+0000","last_peered":"2023-07-08T12:38:13.759477+0000","last_clean":"2023-07-08T12:38:13.759477+0000","last_became_active":"2023-07-08T12:38:08.556016+0000","last_became_peered":"2023-07-08T12:38:08.556016+0000","last_unstale":"2023-07-08T12:38:13.759477+0000","last_undegraded":"2023-07-08T12:38:13.759477+0000","last_fullsized":"2023-07-08T12:38:13.759477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605611+0000","last_change":"2023-07-08T12:38:12.584715+0000","last_active":"2023-07-08T12:38:13.605611+0000","last_peered":"2023-07-08T12:38:13.605611+0000","last_clean":"2023-07-08T12:38:13.605611+0000","last_became_active":"2023-07-08T12:38:12.584249+0000","last_became_peered":"2023-07-08T12:38:12.584249+0000","last_unstale":"2023-07-08T12:38:13.605611+0000","last_undegraded":"2023-07-08T12:38:13.605611+0000","last_fullsized":"2023-07-08T12:38:13.605611+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:34:52.829894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759931+0000","last_change":"2023-07-08T12:38:07.537430+0000","last_active":"2023-07-08T12:38:13.759931+0000","last_peered":"2023-07-08T12:38:13.759931+0000","last_clean":"2023-07-08T12:38:13.759931+0000","last_became_active":"2023-07-08T12:38:07.537043+0000","last_became_peered":"2023-07-08T12:38:07.537043+0000","last_unstale":"2023-07-08T12:38:13.759931+0000","last_undegraded":"2023-07-08T12:38:13.759931+0000","last_fullsized":"2023-07-08T12:38:13.759931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:18:17.308237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760054+0000","last_change":"2023-07-08T12:38:02.385821+0000","last_active":"2023-07-08T12:38:13.760054+0000","last_peered":"2023-07-08T12:38:13.760054+0000","last_clean":"2023-07-08T12:38:13.760054+0000","last_became_active":"2023-07-08T12:38:02.385512+0000","last_became_peered":"2023-07-08T12:38:02.385512+0000","last_unstale":"2023-07-08T12:38:13.760054+0000","last_undegraded":"2023-07-08T12:38:13.760054+0000","last_fullsized":"2023-07-08T12:38:13.760054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:49.796115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758149+0000","last_change":"2023-07-08T12:38:08.556555+0000","last_active":"2023-07-08T12:38:13.758149+0000","last_peered":"2023-07-08T12:38:13.758149+0000","last_clean":"2023-07-08T12:38:13.758149+0000","last_became_active":"2023-07-08T12:38:08.555822+0000","last_became_peered":"2023-07-08T12:38:08.555822+0000","last_unstale":"2023-07-08T12:38:13.758149+0000","last_undegraded":"2023-07-08T12:38:13.758149+0000","last_fullsized":"2023-07-08T12:38:13.758149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:15.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758115+0000","last_change":"2023-07-08T12:38:12.587833+0000","last_active":"2023-07-08T12:38:13.758115+0000","last_peered":"2023-07-08T12:38:13.758115+0000","last_clean":"2023-07-08T12:38:13.758115+0000","last_became_active":"2023-07-08T12:38:12.587465+0000","last_became_peered":"2023-07-08T12:38:12.587465+0000","last_unstale":"2023-07-08T12:38:13.758115+0000","last_undegraded":"2023-07-08T12:38:13.758115+0000","last_fullsized":"2023-07-08T12:38:13.758115+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:32.048081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759111+0000","last_change":"2023-07-08T12:38:02.385388+0000","last_active":"2023-07-08T12:38:13.759111+0000","last_peered":"2023-07-08T12:38:13.759111+0000","last_clean":"2023-07-08T12:38:13.759111+0000","last_became_active":"2023-07-08T12:38:02.385110+0000","last_became_peered":"2023-07-08T12:38:02.385110+0000","last_unstale":"2023-07-08T12:38:13.759111+0000","last_undegraded":"2023-07-08T12:38:13.759111+0000","last_fullsized":"2023-07-08T12:38:13.759111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:52:02.887135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759215+0000","last_change":"2023-07-08T12:38:07.544210+0000","last_active":"2023-07-08T12:38:13.759215+0000","last_peered":"2023-07-08T12:38:13.759215+0000","last_clean":"2023-07-08T12:38:13.759215+0000","last_became_active":"2023-07-08T12:38:07.543870+0000","last_became_peered":"2023-07-08T12:38:07.543870+0000","last_unstale":"2023-07-08T12:38:13.759215+0000","last_undegraded":"2023-07-08T12:38:13.759215+0000","last_fullsized":"2023-07-08T12:38:13.759215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:27.000382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606243+0000","last_change":"2023-07-08T12:38:08.548933+0000","last_active":"2023-07-08T12:38:13.606243+0000","last_peered":"2023-07-08T12:38:13.606243+0000","last_clean":"2023-07-08T12:38:13.606243+0000","last_became_active":"2023-07-08T12:38:08.548655+0000","last_became_peered":"2023-07-08T12:38:08.548655+0000","last_unstale":"2023-07-08T12:38:13.606243+0000","last_undegraded":"2023-07-08T12:38:13.606243+0000","last_fullsized":"2023-07-08T12:38:13.606243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:33:14.696652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759816+0000","last_change":"2023-07-08T12:38:12.597637+0000","last_active":"2023-07-08T12:38:13.759816+0000","last_peered":"2023-07-08T12:38:13.759816+0000","last_clean":"2023-07-08T12:38:13.759816+0000","last_became_active":"2023-07-08T12:38:12.597184+0000","last_became_peered":"2023-07-08T12:38:12.597184+0000","last_unstale":"2023-07-08T12:38:13.759816+0000","last_undegraded":"2023-07-08T12:38:13.759816+0000","last_fullsized":"2023-07-08T12:38:13.759816+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:44.677613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759341+0000","last_change":"2023-07-08T12:38:07.546269+0000","last_active":"2023-07-08T12:38:13.759341+0000","last_peered":"2023-07-08T12:38:13.759341+0000","last_clean":"2023-07-08T12:38:13.759341+0000","last_became_active":"2023-07-08T12:38:07.545942+0000","last_became_peered":"2023-07-08T12:38:07.545942+0000","last_unstale":"2023-07-08T12:38:13.759341+0000","last_undegraded":"2023-07-08T12:38:13.759341+0000","last_fullsized":"2023-07-08T12:38:13.759341+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:55.151640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604575+0000","last_change":"2023-07-08T12:38:08.566129+0000","last_active":"2023-07-08T12:38:13.604575+0000","last_peered":"2023-07-08T12:38:13.604575+0000","last_clean":"2023-07-08T12:38:13.604575+0000","last_became_active":"2023-07-08T12:38:08.565780+0000","last_became_peered":"2023-07-08T12:38:08.565780+0000","last_unstale":"2023-07-08T12:38:13.604575+0000","last_undegraded":"2023-07-08T12:38:13.604575+0000","last_fullsized":"2023-07-08T12:38:13.604575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:52:43.348476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758357+0000","last_change":"2023-07-08T12:38:12.582590+0000","last_active":"2023-07-08T12:38:13.758357+0000","last_peered":"2023-07-08T12:38:13.758357+0000","last_clean":"2023-07-08T12:38:13.758357+0000","last_became_active":"2023-07-08T12:38:12.582179+0000","last_became_peered":"2023-07-08T12:38:12.582179+0000","last_unstale":"2023-07-08T12:38:13.758357+0000","last_undegraded":"2023-07-08T12:38:13.758357+0000","last_fullsized":"2023-07-08T12:38:13.758357+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:39.801225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604721+0000","last_change":"2023-07-08T12:38:07.544462+0000","last_active":"2023-07-08T12:38:13.604721+0000","last_peered":"2023-07-08T12:38:13.604721+0000","last_clean":"2023-07-08T12:38:13.604721+0000","last_became_active":"2023-07-08T12:38:07.544171+0000","last_became_peered":"2023-07-08T12:38:07.544171+0000","last_unstale":"2023-07-08T12:38:13.604721+0000","last_undegraded":"2023-07-08T12:38:13.604721+0000","last_fullsized":"2023-07-08T12:38:13.604721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:14.285892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605091+0000","last_change":"2023-07-08T12:38:08.557546+0000","last_active":"2023-07-08T12:38:13.605091+0000","last_peered":"2023-07-08T12:38:13.605091+0000","last_clean":"2023-07-08T12:38:13.605091+0000","last_became_active":"2023-07-08T12:38:08.557262+0000","last_became_peered":"2023-07-08T12:38:08.557262+0000","last_unstale":"2023-07-08T12:38:13.605091+0000","last_undegraded":"2023-07-08T12:38:13.605091+0000","last_fullsized":"2023-07-08T12:38:13.605091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:11:12.440266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605361+0000","last_change":"2023-07-08T12:38:12.597319+0000","last_active":"2023-07-08T12:38:13.605361+0000","last_peered":"2023-07-08T12:38:13.605361+0000","last_clean":"2023-07-08T12:38:13.605361+0000","last_became_active":"2023-07-08T12:38:12.596928+0000","last_became_peered":"2023-07-08T12:38:12.596928+0000","last_unstale":"2023-07-08T12:38:13.605361+0000","last_undegraded":"2023-07-08T12:38:13.605361+0000","last_fullsized":"2023-07-08T12:38:13.605361+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:39.375709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604658+0000","last_change":"2023-07-08T12:38:07.540676+0000","last_active":"2023-07-08T12:38:13.604658+0000","last_peered":"2023-07-08T12:38:13.604658+0000","last_clean":"2023-07-08T12:38:13.604658+0000","last_became_active":"2023-07-08T12:38:07.540389+0000","last_became_peered":"2023-07-08T12:38:07.540389+0000","last_unstale":"2023-07-08T12:38:13.604658+0000","last_undegraded":"2023-07-08T12:38:13.604658+0000","last_fullsized":"2023-07-08T12:38:13.604658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:27.000382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606089+0000","last_change":"2023-07-08T12:38:08.546186+0000","last_active":"2023-07-08T12:38:13.606089+0000","last_peered":"2023-07-08T12:38:13.606089+0000","last_clean":"2023-07-08T12:38:13.606089+0000","last_became_active":"2023-07-08T12:38:08.545902+0000","last_became_peered":"2023-07-08T12:38:08.545902+0000","last_unstale":"2023-07-08T12:38:13.606089+0000","last_undegraded":"2023-07-08T12:38:13.606089+0000","last_fullsized":"2023-07-08T12:38:13.606089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:37:20.499952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604484+0000","last_change":"2023-07-08T12:38:12.581579+0000","last_active":"2023-07-08T12:38:13.604484+0000","last_peered":"2023-07-08T12:38:13.604484+0000","last_clean":"2023-07-08T12:38:13.604484+0000","last_became_active":"2023-07-08T12:38:12.581118+0000","last_became_peered":"2023-07-08T12:38:12.581118+0000","last_unstale":"2023-07-08T12:38:13.604484+0000","last_undegraded":"2023-07-08T12:38:13.604484+0000","last_fullsized":"2023-07-08T12:38:13.604484+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:48:23.748317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758339+0000","last_change":"2023-07-08T12:38:07.541066+0000","last_active":"2023-07-08T12:38:13.758339+0000","last_peered":"2023-07-08T12:38:13.758339+0000","last_clean":"2023-07-08T12:38:13.758339+0000","last_became_active":"2023-07-08T12:38:07.540873+0000","last_became_peered":"2023-07-08T12:38:07.540873+0000","last_unstale":"2023-07-08T12:38:13.758339+0000","last_undegraded":"2023-07-08T12:38:13.758339+0000","last_fullsized":"2023-07-08T12:38:13.758339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:34.328010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758518+0000","last_change":"2023-07-08T12:38:08.553600+0000","last_active":"2023-07-08T12:38:13.758518+0000","last_peered":"2023-07-08T12:38:13.758518+0000","last_clean":"2023-07-08T12:38:13.758518+0000","last_became_active":"2023-07-08T12:38:08.552418+0000","last_became_peered":"2023-07-08T12:38:08.552418+0000","last_unstale":"2023-07-08T12:38:13.758518+0000","last_undegraded":"2023-07-08T12:38:13.758518+0000","last_fullsized":"2023-07-08T12:38:13.758518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:36:03.651816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758948+0000","last_change":"2023-07-08T12:38:12.590123+0000","last_active":"2023-07-08T12:38:13.758948+0000","last_peered":"2023-07-08T12:38:13.758948+0000","last_clean":"2023-07-08T12:38:13.758948+0000","last_became_active":"2023-07-08T12:38:12.589586+0000","last_became_peered":"2023-07-08T12:38:12.589586+0000","last_unstale":"2023-07-08T12:38:13.758948+0000","last_undegraded":"2023-07-08T12:38:13.758948+0000","last_fullsized":"2023-07-08T12:38:13.758948+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:03:21.918481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759739+0000","last_change":"2023-07-08T12:38:07.541648+0000","last_active":"2023-07-08T12:38:13.759739+0000","last_peered":"2023-07-08T12:38:13.759739+0000","last_clean":"2023-07-08T12:38:13.759739+0000","last_became_active":"2023-07-08T12:38:07.541284+0000","last_became_peered":"2023-07-08T12:38:07.541284+0000","last_unstale":"2023-07-08T12:38:13.759739+0000","last_undegraded":"2023-07-08T12:38:13.759739+0000","last_fullsized":"2023-07-08T12:38:13.759739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:36.285500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606128+0000","last_change":"2023-07-08T12:38:08.552907+0000","last_active":"2023-07-08T12:38:13.606128+0000","last_peered":"2023-07-08T12:38:13.606128+0000","last_clean":"2023-07-08T12:38:13.606128+0000","last_became_active":"2023-07-08T12:38:08.552595+0000","last_became_peered":"2023-07-08T12:38:08.552595+0000","last_unstale":"2023-07-08T12:38:13.606128+0000","last_undegraded":"2023-07-08T12:38:13.606128+0000","last_fullsized":"2023-07-08T12:38:13.606128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:02:09.276408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759378+0000","last_change":"2023-07-08T12:38:08.554379+0000","last_active":"2023-07-08T12:38:13.759378+0000","last_peered":"2023-07-08T12:38:13.759378+0000","last_clean":"2023-07-08T12:38:13.759378+0000","last_became_active":"2023-07-08T12:38:08.554092+0000","last_became_peered":"2023-07-08T12:38:08.554092+0000","last_unstale":"2023-07-08T12:38:13.759378+0000","last_undegraded":"2023-07-08T12:38:13.759378+0000","last_fullsized":"2023-07-08T12:38:13.759378+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:36:56.254060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758785+0000","last_change":"2023-07-08T12:38:08.557447+0000","last_active":"2023-07-08T12:38:13.758785+0000","last_peered":"2023-07-08T12:38:13.758785+0000","last_clean":"2023-07-08T12:38:13.758785+0000","last_became_active":"2023-07-08T12:38:08.557112+0000","last_became_peered":"2023-07-08T12:38:08.557112+0000","last_unstale":"2023-07-08T12:38:13.758785+0000","last_undegraded":"2023-07-08T12:38:13.758785+0000","last_fullsized":"2023-07-08T12:38:13.758785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:31:53.990746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606414+0000","last_change":"2023-07-08T12:38:08.550996+0000","last_active":"2023-07-08T12:38:13.606414+0000","last_peered":"2023-07-08T12:38:13.606414+0000","last_clean":"2023-07-08T12:38:13.606414+0000","last_became_active":"2023-07-08T12:38:08.550597+0000","last_became_peered":"2023-07-08T12:38:08.550597+0000","last_unstale":"2023-07-08T12:38:13.606414+0000","last_undegraded":"2023-07-08T12:38:13.606414+0000","last_fullsized":"2023-07-08T12:38:13.606414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:44:57.917258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758464+0000","last_change":"2023-07-08T12:38:08.552118+0000","last_active":"2023-07-08T12:38:13.758464+0000","last_peered":"2023-07-08T12:38:13.758464+0000","last_clean":"2023-07-08T12:38:13.758464+0000","last_became_active":"2023-07-08T12:38:08.551258+0000","last_became_peered":"2023-07-08T12:38:08.551258+0000","last_unstale":"2023-07-08T12:38:13.758464+0000","last_undegraded":"2023-07-08T12:38:13.758464+0000","last_fullsized":"2023-07-08T12:38:13.758464+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:22.667819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759628+0000","last_change":"2023-07-08T12:38:08.563050+0000","last_active":"2023-07-08T12:38:13.759628+0000","last_peered":"2023-07-08T12:38:13.759628+0000","last_clean":"2023-07-08T12:38:13.759628+0000","last_became_active":"2023-07-08T12:38:08.562779+0000","last_became_peered":"2023-07-08T12:38:08.562779+0000","last_unstale":"2023-07-08T12:38:13.759628+0000","last_undegraded":"2023-07-08T12:38:13.759628+0000","last_fullsized":"2023-07-08T12:38:13.759628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:15.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4592,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":718950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":901120,"data_stored":715522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":323584,"data_stored":128422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":311296,"data_stored":125224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-08T12:38:33.485 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-08T12:38:33.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.633+0000 7f7458244700 1 Processor -- start 2023-07-08T12:38:33.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.633+0000 7f7458244700 1 -- start start 2023-07-08T12:38:33.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.634+0000 7f7458244700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7450160790 0x7f74500cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:33.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.634+0000 7f7458244700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7450069f30 0x7f745006a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:33.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.634+0000 7f7458244700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f745015c890 con 0x7f7450159c30 2023-07-08T12:38:33.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.634+0000 7f7458244700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745015c730 con 0x7f7450160790 2023-07-08T12:38:33.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.634+0000 7f7458244700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74500cf590 con 0x7f7450069f30 2023-07-08T12:38:33.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.634+0000 7f74557df700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7450160790 0x7f74500cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:33.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.634+0000 7f74557df700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7450160790 0x7f74500cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49572/0 (socket says 172.21.15.12:49572) 2023-07-08T12:38:33.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.634+0000 7f74557df700 1 -- 172.21.15.12:0/3948911477 learned_addr learned my addr 172.21.15.12:0/3948911477 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:33.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.634+0000 7f74567e1700 1 --2- 172.21.15.12:0/3948911477 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7450069f30 0x7f745006a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:33.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.634+0000 7f74557df700 1 -- 172.21.15.12:0/3948911477 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7450069f30 msgr2=0x7f745006a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:33.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.634+0000 7f74557df700 1 --2- 172.21.15.12:0/3948911477 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7450069f30 0x7f745006a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.635+0000 7f74557df700 1 -- 172.21.15.12:0/3948911477 >> v1:172.21.15.160:6789/0 conn(0x7f7450159c30 legacy=0x7f745006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.635+0000 7f74557df700 1 -- 172.21.15.12:0/3948911477 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745006f620 con 0x7f7450160790 2023-07-08T12:38:33.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.635+0000 7f74557df700 1 --2- 172.21.15.12:0/3948911477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7450160790 0x7f74500cf050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f744c009920 tx=0x7f744c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d99e7ecb059883 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:33.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.635+0000 7f7454fde700 1 -- 172.21.15.12:0/3948911477 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c00ba20 con 0x7f7450160790 2023-07-08T12:38:33.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.635+0000 7f7454fde700 1 -- 172.21.15.12:0/3948911477 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c00bb80 con 0x7f7450160790 2023-07-08T12:38:33.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.635+0000 7f7454fde700 1 -- 172.21.15.12:0/3948911477 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c009e70 con 0x7f7450160790 2023-07-08T12:38:33.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.635+0000 7f7458244700 1 -- 172.21.15.12:0/3948911477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7450160790 msgr2=0x7f74500cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.635+0000 7f7458244700 1 --2- 172.21.15.12:0/3948911477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7450160790 0x7f74500cf050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f744c009920 tx=0x7f744c009c30 comp rx=0 tx=0).stop 2023-07-08T12:38:33.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.636+0000 7f7458244700 1 -- 172.21.15.12:0/3948911477 shutdown_connections 2023-07-08T12:38:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.636+0000 7f7458244700 1 --2- 172.21.15.12:0/3948911477 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7450069f30 0x7f745006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.636+0000 7f7458244700 1 --2- 172.21.15.12:0/3948911477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7450160790 0x7f74500cf050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.636+0000 7f7458244700 1 -- 172.21.15.12:0/3948911477 >> 172.21.15.12:0/3948911477 conn(0x7f74500cc290 msgr2=0x7f74500cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.636+0000 7f7458244700 1 -- 172.21.15.12:0/3948911477 shutdown_connections 2023-07-08T12:38:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.636+0000 7f7458244700 1 -- 172.21.15.12:0/3948911477 wait complete. 2023-07-08T12:38:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.637+0000 7f7458244700 1 Processor -- start 2023-07-08T12:38:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.637+0000 7f7458244700 1 -- start start 2023-07-08T12:38:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7458244700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7450069f30 0x7f74500b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7458244700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7450159c30 0x7f74500b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7458244700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7450160790 0x7f7450153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7458244700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74500b3050 con 0x7f7450069f30 2023-07-08T12:38:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7458244700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74500b32a0 con 0x7f7450160790 2023-07-08T12:38:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7458244700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74500b3580 con 0x7f7450159c30 2023-07-08T12:38:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7455fe0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7450069f30 0x7f74500b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:33.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7455fe0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7450069f30 0x7f74500b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49576/0 (socket says 172.21.15.12:49576) 2023-07-08T12:38:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7455fe0700 1 -- 172.21.15.12:0/3558655613 learned_addr learned my addr 172.21.15.12:0/3558655613 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f74557df700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7450159c30 0x7f74500b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f74567e1700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7450160790 0x7f7450153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7455fe0700 1 -- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7450159c30 msgr2=0x7f74500b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7455fe0700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7450159c30 0x7f74500b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7455fe0700 1 -- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7450160790 msgr2=0x7f7450153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7455fe0700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7450160790 0x7f7450153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7455fe0700 1 -- 172.21.15.12:0/3558655613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c009550 con 0x7f7450069f30 2023-07-08T12:38:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f74567e1700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7450160790 0x7f7450153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7455fe0700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7450069f30 0x7f74500b4060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f744000c4e0 tx=0x7f744000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f74467fc700 1 -- 172.21.15.12:0/3558655613 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74400052d0 con 0x7f7450069f30 2023-07-08T12:38:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f74467fc700 1 -- 172.21.15.12:0/3558655613 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7440005430 con 0x7f7450069f30 2023-07-08T12:38:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7458244700 1 -- 172.21.15.12:0/3558655613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74500b3860 con 0x7f7450069f30 2023-07-08T12:38:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f74467fc700 1 -- 172.21.15.12:0/3558655613 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74400038c0 con 0x7f7450069f30 2023-07-08T12:38:33.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.638+0000 7f7458244700 1 -- 172.21.15.12:0/3558655613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74500b45a0 con 0x7f7450069f30 2023-07-08T12:38:33.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.640+0000 7f7458244700 1 -- 172.21.15.12:0/3558655613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74500b3d80 con 0x7f7450069f30 2023-07-08T12:38:33.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.640+0000 7f74467fc700 1 -- 172.21.15.12:0/3558655613 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7440003a20 con 0x7f7450069f30 2023-07-08T12:38:33.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.640+0000 7f74467fc700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f743406ace0 0x7f743406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:33.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.640+0000 7f74467fc700 1 -- 172.21.15.12:0/3558655613 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7440088bf0 con 0x7f7450069f30 2023-07-08T12:38:33.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.641+0000 7f74557df700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f743406ace0 0x7f743406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:33.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.641+0000 7f74557df700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f743406ace0 0x7f743406d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7450160bc0 tx=0x7f744c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:33.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.645+0000 7f74467fc700 1 -- 172.21.15.12:0/3558655613 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f74500b3d80 con 0x7f7450069f30 2023-07-08T12:38:33.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.841+0000 7f7458244700 1 -- 172.21.15.12:0/3558655613 --> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7450160c90 con 0x7f743406ace0 2023-07-08T12:38:33.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.851+0000 7f74467fc700 1 -- 172.21.15.12:0/3558655613 <== mgr.4103 v2:172.21.15.160:6832/550875184 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367308 (secure 0 0 0) 0x7f744c00b820 con 0x7f743406ace0 2023-07-08T12:38:33.858 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.855+0000 7f7458244700 1 -- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f743406ace0 msgr2=0x7f743406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.855+0000 7f7458244700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f743406ace0 0x7f743406d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7450160bc0 tx=0x7f744c008030 comp rx=0 tx=0).stop 2023-07-08T12:38:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.855+0000 7f7458244700 1 -- 172.21.15.12:0/3558655613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7450069f30 msgr2=0x7f74500b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.855+0000 7f7458244700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7450069f30 0x7f74500b4060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f744000c4e0 tx=0x7f744000c8a0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.855+0000 7f7458244700 1 -- 172.21.15.12:0/3558655613 shutdown_connections 2023-07-08T12:38:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.855+0000 7f7458244700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f743406ace0 0x7f743406d190 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.855+0000 7f7458244700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7450160790 0x7f7450153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.855+0000 7f7458244700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7450069f30 0x7f74500b4060 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.855+0000 7f7458244700 1 --2- 172.21.15.12:0/3558655613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7450159c30 0x7f74500b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.855+0000 7f7458244700 1 -- 172.21.15.12:0/3558655613 >> 172.21.15.12:0/3558655613 conn(0x7f74500cc290 msgr2=0x7f74500aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.855+0000 7f7458244700 1 -- 172.21.15.12:0/3558655613 shutdown_connections 2023-07-08T12:38:33.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:33.855+0000 7f7458244700 1 -- 172.21.15.12:0/3558655613 wait complete. 2023-07-08T12:38:33.866 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-07-08T12:38:33.868 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-07-08T12:38:32.145661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594960,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219208,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755512,"statfs":{"total":773094113280,"available":772869644288,"internally_reserved":0,"allocated":3678208,"data_stored":2186496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003442"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604481+0000","last_change":"2023-07-08T12:38:08.558253+0000","last_active":"2023-07-08T12:38:13.604481+0000","last_peered":"2023-07-08T12:38:13.604481+0000","last_clean":"2023-07-08T12:38:13.604481+0000","last_became_active":"2023-07-08T12:38:08.557970+0000","last_became_peered":"2023-07-08T12:38:08.557970+0000","last_unstale":"2023-07-08T12:38:13.604481+0000","last_undegraded":"2023-07-08T12:38:13.604481+0000","last_fullsized":"2023-07-08T12:38:13.604481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:56:17.613729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.608666+0000","last_change":"2023-07-08T12:38:08.546715+0000","last_active":"2023-07-08T12:38:13.608666+0000","last_peered":"2023-07-08T12:38:13.608666+0000","last_clean":"2023-07-08T12:38:13.608666+0000","last_became_active":"2023-07-08T12:38:08.546365+0000","last_became_peered":"2023-07-08T12:38:08.546365+0000","last_unstale":"2023-07-08T12:38:13.608666+0000","last_undegraded":"2023-07-08T12:38:13.608666+0000","last_fullsized":"2023-07-08T12:38:13.608666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:55:50.492319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606777+0000","last_change":"2023-07-08T12:38:08.561903+0000","last_active":"2023-07-08T12:38:13.606777+0000","last_peered":"2023-07-08T12:38:13.606777+0000","last_clean":"2023-07-08T12:38:13.606777+0000","last_became_active":"2023-07-08T12:38:08.561533+0000","last_became_peered":"2023-07-08T12:38:08.561533+0000","last_unstale":"2023-07-08T12:38:13.606777+0000","last_undegraded":"2023-07-08T12:38:13.606777+0000","last_fullsized":"2023-07-08T12:38:13.606777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:26:30.519413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758858+0000","last_change":"2023-07-08T12:38:08.556755+0000","last_active":"2023-07-08T12:38:13.758858+0000","last_peered":"2023-07-08T12:38:13.758858+0000","last_clean":"2023-07-08T12:38:13.758858+0000","last_became_active":"2023-07-08T12:38:08.556476+0000","last_became_peered":"2023-07-08T12:38:08.556476+0000","last_unstale":"2023-07-08T12:38:13.758858+0000","last_undegraded":"2023-07-08T12:38:13.758858+0000","last_fullsized":"2023-07-08T12:38:13.758858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:52.854234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759575+0000","last_change":"2023-07-08T12:38:08.561313+0000","last_active":"2023-07-08T12:38:13.759575+0000","last_peered":"2023-07-08T12:38:13.759575+0000","last_clean":"2023-07-08T12:38:13.759575+0000","last_became_active":"2023-07-08T12:38:08.561055+0000","last_became_peered":"2023-07-08T12:38:08.561055+0000","last_unstale":"2023-07-08T12:38:13.759575+0000","last_undegraded":"2023-07-08T12:38:13.759575+0000","last_fullsized":"2023-07-08T12:38:13.759575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:49.502059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606630+0000","last_change":"2023-07-08T12:38:08.564651+0000","last_active":"2023-07-08T12:38:13.606630+0000","last_peered":"2023-07-08T12:38:13.606630+0000","last_clean":"2023-07-08T12:38:13.606630+0000","last_became_active":"2023-07-08T12:38:08.564389+0000","last_became_peered":"2023-07-08T12:38:08.564389+0000","last_unstale":"2023-07-08T12:38:13.606630+0000","last_undegraded":"2023-07-08T12:38:13.606630+0000","last_fullsized":"2023-07-08T12:38:13.606630+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:03:39.903947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758619+0000","last_change":"2023-07-08T12:38:08.569425+0000","last_active":"2023-07-08T12:38:13.758619+0000","last_peered":"2023-07-08T12:38:13.758619+0000","last_clean":"2023-07-08T12:38:13.758619+0000","last_became_active":"2023-07-08T12:38:08.569167+0000","last_became_peered":"2023-07-08T12:38:08.569167+0000","last_unstale":"2023-07-08T12:38:13.758619+0000","last_undegraded":"2023-07-08T12:38:13.758619+0000","last_fullsized":"2023-07-08T12:38:13.758619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:07.081181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605308+0000","last_change":"2023-07-08T12:38:08.549000+0000","last_active":"2023-07-08T12:38:13.605308+0000","last_peered":"2023-07-08T12:38:13.605308+0000","last_clean":"2023-07-08T12:38:13.605308+0000","last_became_active":"2023-07-08T12:38:08.548657+0000","last_became_peered":"2023-07-08T12:38:08.548657+0000","last_unstale":"2023-07-08T12:38:13.605308+0000","last_undegraded":"2023-07-08T12:38:13.605308+0000","last_fullsized":"2023-07-08T12:38:13.605308+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:37.290100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759490+0000","last_change":"2023-07-08T12:38:08.565002+0000","last_active":"2023-07-08T12:38:13.759490+0000","last_peered":"2023-07-08T12:38:13.759490+0000","last_clean":"2023-07-08T12:38:13.759490+0000","last_became_active":"2023-07-08T12:38:08.564554+0000","last_became_peered":"2023-07-08T12:38:08.564554+0000","last_unstale":"2023-07-08T12:38:13.759490+0000","last_undegraded":"2023-07-08T12:38:13.759490+0000","last_fullsized":"2023-07-08T12:38:13.759490+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:35.758126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606670+0000","last_change":"2023-07-08T12:38:08.556118+0000","last_active":"2023-07-08T12:38:13.606670+0000","last_peered":"2023-07-08T12:38:13.606670+0000","last_clean":"2023-07-08T12:38:13.606670+0000","last_became_active":"2023-07-08T12:38:08.555833+0000","last_became_peered":"2023-07-08T12:38:08.555833+0000","last_unstale":"2023-07-08T12:38:13.606670+0000","last_undegraded":"2023-07-08T12:38:13.606670+0000","last_fullsized":"2023-07-08T12:38:13.606670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:03.390028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606704+0000","last_change":"2023-07-08T12:38:08.568872+0000","last_active":"2023-07-08T12:38:13.606704+0000","last_peered":"2023-07-08T12:38:13.606704+0000","last_clean":"2023-07-08T12:38:13.606704+0000","last_became_active":"2023-07-08T12:38:08.568624+0000","last_became_peered":"2023-07-08T12:38:08.568624+0000","last_unstale":"2023-07-08T12:38:13.606704+0000","last_undegraded":"2023-07-08T12:38:13.606704+0000","last_fullsized":"2023-07-08T12:38:13.606704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:35:09.678657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604156+0000","last_change":"2023-07-08T12:38:08.560950+0000","last_active":"2023-07-08T12:38:13.604156+0000","last_peered":"2023-07-08T12:38:13.604156+0000","last_clean":"2023-07-08T12:38:13.604156+0000","last_became_active":"2023-07-08T12:38:08.560621+0000","last_became_peered":"2023-07-08T12:38:08.560621+0000","last_unstale":"2023-07-08T12:38:13.604156+0000","last_undegraded":"2023-07-08T12:38:13.604156+0000","last_fullsized":"2023-07-08T12:38:13.604156+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T12:59:32.693976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604193+0000","last_change":"2023-07-08T12:38:08.556685+0000","last_active":"2023-07-08T12:38:13.604193+0000","last_peered":"2023-07-08T12:38:13.604193+0000","last_clean":"2023-07-08T12:38:13.604193+0000","last_became_active":"2023-07-08T12:38:08.556387+0000","last_became_peered":"2023-07-08T12:38:08.556387+0000","last_unstale":"2023-07-08T12:38:13.604193+0000","last_undegraded":"2023-07-08T12:38:13.604193+0000","last_fullsized":"2023-07-08T12:38:13.604193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:03:36.626338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604952+0000","last_change":"2023-07-08T12:38:08.547070+0000","last_active":"2023-07-08T12:38:13.604952+0000","last_peered":"2023-07-08T12:38:13.604952+0000","last_clean":"2023-07-08T12:38:13.604952+0000","last_became_active":"2023-07-08T12:38:08.546813+0000","last_became_peered":"2023-07-08T12:38:08.546813+0000","last_unstale":"2023-07-08T12:38:13.604952+0000","last_undegraded":"2023-07-08T12:38:13.604952+0000","last_fullsized":"2023-07-08T12:38:13.604952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758137+0000","last_change":"2023-07-08T12:38:08.569414+0000","last_active":"2023-07-08T12:38:13.758137+0000","last_peered":"2023-07-08T12:38:13.758137+0000","last_clean":"2023-07-08T12:38:13.758137+0000","last_became_active":"2023-07-08T12:38:08.569177+0000","last_became_peered":"2023-07-08T12:38:08.569177+0000","last_unstale":"2023-07-08T12:38:13.758137+0000","last_undegraded":"2023-07-08T12:38:13.758137+0000","last_fullsized":"2023-07-08T12:38:13.758137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606000+0000","last_change":"2023-07-08T12:38:08.561730+0000","last_active":"2023-07-08T12:38:13.606000+0000","last_peered":"2023-07-08T12:38:13.606000+0000","last_clean":"2023-07-08T12:38:13.606000+0000","last_became_active":"2023-07-08T12:38:08.561498+0000","last_became_peered":"2023-07-08T12:38:08.561498+0000","last_unstale":"2023-07-08T12:38:13.606000+0000","last_undegraded":"2023-07-08T12:38:13.606000+0000","last_fullsized":"2023-07-08T12:38:13.606000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:06:06.921530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759585+0000","last_change":"2023-07-08T12:38:08.568400+0000","last_active":"2023-07-08T12:38:13.759585+0000","last_peered":"2023-07-08T12:38:13.759585+0000","last_clean":"2023-07-08T12:38:13.759585+0000","last_became_active":"2023-07-08T12:38:08.568070+0000","last_became_peered":"2023-07-08T12:38:08.568070+0000","last_unstale":"2023-07-08T12:38:13.759585+0000","last_undegraded":"2023-07-08T12:38:13.759585+0000","last_fullsized":"2023-07-08T12:38:13.759585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:35.332610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606886+0000","last_change":"2023-07-08T12:38:08.549069+0000","last_active":"2023-07-08T12:38:13.606886+0000","last_peered":"2023-07-08T12:38:13.606886+0000","last_clean":"2023-07-08T12:38:13.606886+0000","last_became_active":"2023-07-08T12:38:08.548774+0000","last_became_peered":"2023-07-08T12:38:08.548774+0000","last_unstale":"2023-07-08T12:38:13.606886+0000","last_undegraded":"2023-07-08T12:38:13.606886+0000","last_fullsized":"2023-07-08T12:38:13.606886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:02:05.917920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604270+0000","last_change":"2023-07-08T12:38:08.553458+0000","last_active":"2023-07-08T12:38:13.604270+0000","last_peered":"2023-07-08T12:38:13.604270+0000","last_clean":"2023-07-08T12:38:13.604270+0000","last_became_active":"2023-07-08T12:38:08.553177+0000","last_became_peered":"2023-07-08T12:38:08.553177+0000","last_unstale":"2023-07-08T12:38:13.604270+0000","last_undegraded":"2023-07-08T12:38:13.604270+0000","last_fullsized":"2023-07-08T12:38:13.604270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:07:08.413767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606438+0000","last_change":"2023-07-08T12:38:08.546039+0000","last_active":"2023-07-08T12:38:13.606438+0000","last_peered":"2023-07-08T12:38:13.606438+0000","last_clean":"2023-07-08T12:38:13.606438+0000","last_became_active":"2023-07-08T12:38:08.545723+0000","last_became_peered":"2023-07-08T12:38:08.545723+0000","last_unstale":"2023-07-08T12:38:13.606438+0000","last_undegraded":"2023-07-08T12:38:13.606438+0000","last_fullsized":"2023-07-08T12:38:13.606438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:05:03.136361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758578+0000","last_change":"2023-07-08T12:38:08.558172+0000","last_active":"2023-07-08T12:38:13.758578+0000","last_peered":"2023-07-08T12:38:13.758578+0000","last_clean":"2023-07-08T12:38:13.758578+0000","last_became_active":"2023-07-08T12:38:08.557888+0000","last_became_peered":"2023-07-08T12:38:08.557888+0000","last_unstale":"2023-07-08T12:38:13.758578+0000","last_undegraded":"2023-07-08T12:38:13.758578+0000","last_fullsized":"2023-07-08T12:38:13.758578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:37.290100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758835+0000","last_change":"2023-07-08T12:38:08.569776+0000","last_active":"2023-07-08T12:38:13.758835+0000","last_peered":"2023-07-08T12:38:13.758835+0000","last_clean":"2023-07-08T12:38:13.758835+0000","last_became_active":"2023-07-08T12:38:08.569530+0000","last_became_peered":"2023-07-08T12:38:08.569530+0000","last_unstale":"2023-07-08T12:38:13.758835+0000","last_undegraded":"2023-07-08T12:38:13.758835+0000","last_fullsized":"2023-07-08T12:38:13.758835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:18:27.333138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759672+0000","last_change":"2023-07-08T12:38:08.557720+0000","last_active":"2023-07-08T12:38:13.759672+0000","last_peered":"2023-07-08T12:38:13.759672+0000","last_clean":"2023-07-08T12:38:13.759672+0000","last_became_active":"2023-07-08T12:38:08.557390+0000","last_became_peered":"2023-07-08T12:38:08.557390+0000","last_unstale":"2023-07-08T12:38:13.759672+0000","last_undegraded":"2023-07-08T12:38:13.759672+0000","last_fullsized":"2023-07-08T12:38:13.759672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:13:13.582721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758123+0000","last_change":"2023-07-08T12:38:08.558048+0000","last_active":"2023-07-08T12:38:13.758123+0000","last_peered":"2023-07-08T12:38:13.758123+0000","last_clean":"2023-07-08T12:38:13.758123+0000","last_became_active":"2023-07-08T12:38:08.557871+0000","last_became_peered":"2023-07-08T12:38:08.557871+0000","last_unstale":"2023-07-08T12:38:13.758123+0000","last_undegraded":"2023-07-08T12:38:13.758123+0000","last_fullsized":"2023-07-08T12:38:13.758123+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:40.634514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759316+0000","last_change":"2023-07-08T12:38:08.553223+0000","last_active":"2023-07-08T12:38:13.759316+0000","last_peered":"2023-07-08T12:38:13.759316+0000","last_clean":"2023-07-08T12:38:13.759316+0000","last_became_active":"2023-07-08T12:38:08.552416+0000","last_became_peered":"2023-07-08T12:38:08.552416+0000","last_unstale":"2023-07-08T12:38:13.759316+0000","last_undegraded":"2023-07-08T12:38:13.759316+0000","last_fullsized":"2023-07-08T12:38:13.759316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:06:26.551781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760263+0000","last_change":"2023-07-08T12:38:08.553769+0000","last_active":"2023-07-08T12:38:13.760263+0000","last_peered":"2023-07-08T12:38:13.760263+0000","last_clean":"2023-07-08T12:38:13.760263+0000","last_became_active":"2023-07-08T12:38:08.553502+0000","last_became_peered":"2023-07-08T12:38:08.553502+0000","last_unstale":"2023-07-08T12:38:13.760263+0000","last_undegraded":"2023-07-08T12:38:13.760263+0000","last_fullsized":"2023-07-08T12:38:13.760263+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:35.758126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605466+0000","last_change":"2023-07-08T12:38:08.546308+0000","last_active":"2023-07-08T12:38:13.605466+0000","last_peered":"2023-07-08T12:38:13.605466+0000","last_clean":"2023-07-08T12:38:13.605466+0000","last_became_active":"2023-07-08T12:38:08.545865+0000","last_became_peered":"2023-07-08T12:38:08.545865+0000","last_unstale":"2023-07-08T12:38:13.605466+0000","last_undegraded":"2023-07-08T12:38:13.605466+0000","last_fullsized":"2023-07-08T12:38:13.605466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:22.667819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606681+0000","last_change":"2023-07-08T12:38:12.584534+0000","last_active":"2023-07-08T12:38:13.606681+0000","last_peered":"2023-07-08T12:38:13.606681+0000","last_clean":"2023-07-08T12:38:13.606681+0000","last_became_active":"2023-07-08T12:38:12.584153+0000","last_became_peered":"2023-07-08T12:38:12.584153+0000","last_unstale":"2023-07-08T12:38:13.606681+0000","last_undegraded":"2023-07-08T12:38:13.606681+0000","last_fullsized":"2023-07-08T12:38:13.606681+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:21:34.996680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606263+0000","last_change":"2023-07-08T12:38:07.547338+0000","last_active":"2023-07-08T12:38:13.606263+0000","last_peered":"2023-07-08T12:38:13.606263+0000","last_clean":"2023-07-08T12:38:13.606263+0000","last_became_active":"2023-07-08T12:38:07.547161+0000","last_became_peered":"2023-07-08T12:38:07.547161+0000","last_unstale":"2023-07-08T12:38:13.606263+0000","last_undegraded":"2023-07-08T12:38:13.606263+0000","last_fullsized":"2023-07-08T12:38:13.606263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:37:52.614496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759146+0000","last_change":"2023-07-08T12:38:08.553623+0000","last_active":"2023-07-08T12:38:13.759146+0000","last_peered":"2023-07-08T12:38:13.759146+0000","last_clean":"2023-07-08T12:38:13.759146+0000","last_became_active":"2023-07-08T12:38:08.551973+0000","last_became_peered":"2023-07-08T12:38:08.551973+0000","last_unstale":"2023-07-08T12:38:13.759146+0000","last_undegraded":"2023-07-08T12:38:13.759146+0000","last_fullsized":"2023-07-08T12:38:13.759146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:55:50.492319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759324+0000","last_change":"2023-07-08T12:38:12.590297+0000","last_active":"2023-07-08T12:38:13.759324+0000","last_peered":"2023-07-08T12:38:13.759324+0000","last_clean":"2023-07-08T12:38:13.759324+0000","last_became_active":"2023-07-08T12:38:12.589898+0000","last_became_peered":"2023-07-08T12:38:12.589898+0000","last_unstale":"2023-07-08T12:38:13.759324+0000","last_undegraded":"2023-07-08T12:38:13.759324+0000","last_fullsized":"2023-07-08T12:38:13.759324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:55.150008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758590+0000","last_change":"2023-07-08T12:38:07.548760+0000","last_active":"2023-07-08T12:38:13.758590+0000","last_peered":"2023-07-08T12:38:13.758590+0000","last_clean":"2023-07-08T12:38:13.758590+0000","last_became_active":"2023-07-08T12:38:07.548448+0000","last_became_peered":"2023-07-08T12:38:07.548448+0000","last_unstale":"2023-07-08T12:38:13.758590+0000","last_undegraded":"2023-07-08T12:38:13.758590+0000","last_fullsized":"2023-07-08T12:38:13.758590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T16:04:24.797809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759344+0000","last_change":"2023-07-08T12:38:08.561715+0000","last_active":"2023-07-08T12:38:13.759344+0000","last_peered":"2023-07-08T12:38:13.759344+0000","last_clean":"2023-07-08T12:38:13.759344+0000","last_became_active":"2023-07-08T12:38:08.561004+0000","last_became_peered":"2023-07-08T12:38:08.561004+0000","last_unstale":"2023-07-08T12:38:13.759344+0000","last_undegraded":"2023-07-08T12:38:13.759344+0000","last_fullsized":"2023-07-08T12:38:13.759344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:40.634514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759948+0000","last_change":"2023-07-08T12:38:12.593094+0000","last_active":"2023-07-08T12:38:13.759948+0000","last_peered":"2023-07-08T12:38:13.759948+0000","last_clean":"2023-07-08T12:38:13.759948+0000","last_became_active":"2023-07-08T12:38:12.592704+0000","last_became_peered":"2023-07-08T12:38:12.592704+0000","last_unstale":"2023-07-08T12:38:13.759948+0000","last_undegraded":"2023-07-08T12:38:13.759948+0000","last_fullsized":"2023-07-08T12:38:13.759948+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:26.710918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758339+0000","last_change":"2023-07-08T12:38:07.543437+0000","last_active":"2023-07-08T12:38:13.758339+0000","last_peered":"2023-07-08T12:38:13.758339+0000","last_clean":"2023-07-08T12:38:13.758339+0000","last_became_active":"2023-07-08T12:38:07.543158+0000","last_became_peered":"2023-07-08T12:38:07.543158+0000","last_unstale":"2023-07-08T12:38:13.758339+0000","last_undegraded":"2023-07-08T12:38:13.758339+0000","last_fullsized":"2023-07-08T12:38:13.758339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:48.497459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759270+0000","last_change":"2023-07-08T12:38:08.552907+0000","last_active":"2023-07-08T12:38:13.759270+0000","last_peered":"2023-07-08T12:38:13.759270+0000","last_clean":"2023-07-08T12:38:13.759270+0000","last_became_active":"2023-07-08T12:38:08.552286+0000","last_became_peered":"2023-07-08T12:38:08.552286+0000","last_unstale":"2023-07-08T12:38:13.759270+0000","last_undegraded":"2023-07-08T12:38:13.759270+0000","last_fullsized":"2023-07-08T12:38:13.759270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:11:32.912205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758741+0000","last_change":"2023-07-08T12:38:12.590030+0000","last_active":"2023-07-08T12:38:13.758741+0000","last_peered":"2023-07-08T12:38:13.758741+0000","last_clean":"2023-07-08T12:38:13.758741+0000","last_became_active":"2023-07-08T12:38:12.589580+0000","last_became_peered":"2023-07-08T12:38:12.589580+0000","last_unstale":"2023-07-08T12:38:13.758741+0000","last_undegraded":"2023-07-08T12:38:13.758741+0000","last_fullsized":"2023-07-08T12:38:13.758741+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:08:35.080577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759616+0000","last_change":"2023-07-08T12:38:07.541564+0000","last_active":"2023-07-08T12:38:13.759616+0000","last_peered":"2023-07-08T12:38:13.759616+0000","last_clean":"2023-07-08T12:38:13.759616+0000","last_became_active":"2023-07-08T12:38:07.540886+0000","last_became_peered":"2023-07-08T12:38:07.540886+0000","last_unstale":"2023-07-08T12:38:13.759616+0000","last_undegraded":"2023-07-08T12:38:13.759616+0000","last_fullsized":"2023-07-08T12:38:13.759616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:35.414408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758276+0000","last_change":"2023-07-08T12:38:08.556742+0000","last_active":"2023-07-08T12:38:13.758276+0000","last_peered":"2023-07-08T12:38:13.758276+0000","last_clean":"2023-07-08T12:38:13.758276+0000","last_became_active":"2023-07-08T12:38:08.556264+0000","last_became_peered":"2023-07-08T12:38:08.556264+0000","last_unstale":"2023-07-08T12:38:13.758276+0000","last_undegraded":"2023-07-08T12:38:13.758276+0000","last_fullsized":"2023-07-08T12:38:13.758276+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:56.156240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758009+0000","last_change":"2023-07-08T12:38:12.587899+0000","last_active":"2023-07-08T12:38:13.758009+0000","last_peered":"2023-07-08T12:38:13.758009+0000","last_clean":"2023-07-08T12:38:13.758009+0000","last_became_active":"2023-07-08T12:38:12.587591+0000","last_became_peered":"2023-07-08T12:38:12.587591+0000","last_unstale":"2023-07-08T12:38:13.758009+0000","last_undegraded":"2023-07-08T12:38:13.758009+0000","last_fullsized":"2023-07-08T12:38:13.758009+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:19.333590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605565+0000","last_change":"2023-07-08T12:38:07.536496+0000","last_active":"2023-07-08T12:38:13.605565+0000","last_peered":"2023-07-08T12:38:13.605565+0000","last_clean":"2023-07-08T12:38:13.605565+0000","last_became_active":"2023-07-08T12:38:07.535995+0000","last_became_peered":"2023-07-08T12:38:07.535995+0000","last_unstale":"2023-07-08T12:38:13.605565+0000","last_undegraded":"2023-07-08T12:38:13.605565+0000","last_fullsized":"2023-07-08T12:38:13.605565+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:28:48.361016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758283+0000","last_change":"2023-07-08T12:38:08.553479+0000","last_active":"2023-07-08T12:38:13.758283+0000","last_peered":"2023-07-08T12:38:13.758283+0000","last_clean":"2023-07-08T12:38:13.758283+0000","last_became_active":"2023-07-08T12:38:08.550664+0000","last_became_peered":"2023-07-08T12:38:08.550664+0000","last_unstale":"2023-07-08T12:38:13.758283+0000","last_undegraded":"2023-07-08T12:38:13.758283+0000","last_fullsized":"2023-07-08T12:38:13.758283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758974+0000","last_change":"2023-07-08T12:38:12.592991+0000","last_active":"2023-07-08T12:38:13.758974+0000","last_peered":"2023-07-08T12:38:13.758974+0000","last_clean":"2023-07-08T12:38:13.758974+0000","last_became_active":"2023-07-08T12:38:12.591995+0000","last_became_peered":"2023-07-08T12:38:12.591995+0000","last_unstale":"2023-07-08T12:38:13.758974+0000","last_undegraded":"2023-07-08T12:38:13.758974+0000","last_fullsized":"2023-07-08T12:38:13.758974+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:07.433127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759097+0000","last_change":"2023-07-08T12:38:07.551354+0000","last_active":"2023-07-08T12:38:13.759097+0000","last_peered":"2023-07-08T12:38:13.759097+0000","last_clean":"2023-07-08T12:38:13.759097+0000","last_became_active":"2023-07-08T12:38:07.551134+0000","last_became_peered":"2023-07-08T12:38:07.551134+0000","last_unstale":"2023-07-08T12:38:13.759097+0000","last_undegraded":"2023-07-08T12:38:13.759097+0000","last_fullsized":"2023-07-08T12:38:13.759097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:41:53.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760978+0000","last_change":"2023-07-08T12:38:08.558470+0000","last_active":"2023-07-08T12:38:13.760978+0000","last_peered":"2023-07-08T12:38:13.760978+0000","last_clean":"2023-07-08T12:38:13.760978+0000","last_became_active":"2023-07-08T12:38:08.557894+0000","last_became_peered":"2023-07-08T12:38:08.557894+0000","last_unstale":"2023-07-08T12:38:13.760978+0000","last_undegraded":"2023-07-08T12:38:13.760978+0000","last_fullsized":"2023-07-08T12:38:13.760978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:35.332610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.757971+0000","last_change":"2023-07-08T12:38:12.590703+0000","last_active":"2023-07-08T12:38:13.757971+0000","last_peered":"2023-07-08T12:38:13.757971+0000","last_clean":"2023-07-08T12:38:13.757971+0000","last_became_active":"2023-07-08T12:38:12.590268+0000","last_became_peered":"2023-07-08T12:38:12.590268+0000","last_unstale":"2023-07-08T12:38:13.757971+0000","last_undegraded":"2023-07-08T12:38:13.757971+0000","last_fullsized":"2023-07-08T12:38:13.757971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:35:36.579637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:23.855571+0000","last_change":"2023-07-08T12:38:07.544325+0000","last_active":"2023-07-08T12:38:23.855571+0000","last_peered":"2023-07-08T12:38:23.855571+0000","last_clean":"2023-07-08T12:38:23.855571+0000","last_became_active":"2023-07-08T12:38:07.543861+0000","last_became_peered":"2023-07-08T12:38:07.543861+0000","last_unstale":"2023-07-08T12:38:23.855571+0000","last_undegraded":"2023-07-08T12:38:23.855571+0000","last_fullsized":"2023-07-08T12:38:23.855571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:40:52.689398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605326+0000","last_change":"2023-07-08T12:38:08.553055+0000","last_active":"2023-07-08T12:38:13.605326+0000","last_peered":"2023-07-08T12:38:13.605326+0000","last_clean":"2023-07-08T12:38:13.605326+0000","last_became_active":"2023-07-08T12:38:08.552632+0000","last_became_peered":"2023-07-08T12:38:08.552632+0000","last_unstale":"2023-07-08T12:38:13.605326+0000","last_undegraded":"2023-07-08T12:38:13.605326+0000","last_fullsized":"2023-07-08T12:38:13.605326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:37:47.885139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605700+0000","last_change":"2023-07-08T12:38:12.598190+0000","last_active":"2023-07-08T12:38:13.605700+0000","last_peered":"2023-07-08T12:38:13.605700+0000","last_clean":"2023-07-08T12:38:13.605700+0000","last_became_active":"2023-07-08T12:38:12.597813+0000","last_became_peered":"2023-07-08T12:38:12.597813+0000","last_unstale":"2023-07-08T12:38:13.605700+0000","last_undegraded":"2023-07-08T12:38:13.605700+0000","last_fullsized":"2023-07-08T12:38:13.605700+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:41:58.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758412+0000","last_change":"2023-07-08T12:38:07.543441+0000","last_active":"2023-07-08T12:38:13.758412+0000","last_peered":"2023-07-08T12:38:13.758412+0000","last_clean":"2023-07-08T12:38:13.758412+0000","last_became_active":"2023-07-08T12:38:07.543188+0000","last_became_peered":"2023-07-08T12:38:07.543188+0000","last_unstale":"2023-07-08T12:38:13.758412+0000","last_undegraded":"2023-07-08T12:38:13.758412+0000","last_fullsized":"2023-07-08T12:38:13.758412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:34.328010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758440+0000","last_change":"2023-07-08T12:38:08.552797+0000","last_active":"2023-07-08T12:38:13.758440+0000","last_peered":"2023-07-08T12:38:13.758440+0000","last_clean":"2023-07-08T12:38:13.758440+0000","last_became_active":"2023-07-08T12:38:08.552289+0000","last_became_peered":"2023-07-08T12:38:08.552289+0000","last_unstale":"2023-07-08T12:38:13.758440+0000","last_undegraded":"2023-07-08T12:38:13.758440+0000","last_fullsized":"2023-07-08T12:38:13.758440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:07.081181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758345+0000","last_change":"2023-07-08T12:38:12.592793+0000","last_active":"2023-07-08T12:38:13.758345+0000","last_peered":"2023-07-08T12:38:13.758345+0000","last_clean":"2023-07-08T12:38:13.758345+0000","last_became_active":"2023-07-08T12:38:12.592414+0000","last_became_peered":"2023-07-08T12:38:12.592414+0000","last_unstale":"2023-07-08T12:38:13.758345+0000","last_undegraded":"2023-07-08T12:38:13.758345+0000","last_fullsized":"2023-07-08T12:38:13.758345+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:39.801225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758616+0000","last_change":"2023-07-08T12:38:07.538612+0000","last_active":"2023-07-08T12:38:13.758616+0000","last_peered":"2023-07-08T12:38:13.758616+0000","last_clean":"2023-07-08T12:38:13.758616+0000","last_became_active":"2023-07-08T12:38:07.538048+0000","last_became_peered":"2023-07-08T12:38:07.538048+0000","last_unstale":"2023-07-08T12:38:13.758616+0000","last_undegraded":"2023-07-08T12:38:13.758616+0000","last_fullsized":"2023-07-08T12:38:13.758616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:04:41.955662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605266+0000","last_change":"2023-07-08T12:38:08.546131+0000","last_active":"2023-07-08T12:38:13.605266+0000","last_peered":"2023-07-08T12:38:13.605266+0000","last_clean":"2023-07-08T12:38:13.605266+0000","last_became_active":"2023-07-08T12:38:08.545491+0000","last_became_peered":"2023-07-08T12:38:08.545491+0000","last_unstale":"2023-07-08T12:38:13.605266+0000","last_undegraded":"2023-07-08T12:38:13.605266+0000","last_fullsized":"2023-07-08T12:38:13.605266+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:36.419008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758428+0000","last_change":"2023-07-08T12:38:12.584683+0000","last_active":"2023-07-08T12:38:13.758428+0000","last_peered":"2023-07-08T12:38:13.758428+0000","last_clean":"2023-07-08T12:38:13.758428+0000","last_became_active":"2023-07-08T12:38:12.584256+0000","last_became_peered":"2023-07-08T12:38:12.584256+0000","last_unstale":"2023-07-08T12:38:13.758428+0000","last_undegraded":"2023-07-08T12:38:13.758428+0000","last_fullsized":"2023-07-08T12:38:13.758428+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:41.333199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758406+0000","last_change":"2023-07-08T12:38:07.543388+0000","last_active":"2023-07-08T12:38:13.758406+0000","last_peered":"2023-07-08T12:38:13.758406+0000","last_clean":"2023-07-08T12:38:13.758406+0000","last_became_active":"2023-07-08T12:38:07.543117+0000","last_became_peered":"2023-07-08T12:38:07.543117+0000","last_unstale":"2023-07-08T12:38:13.758406+0000","last_undegraded":"2023-07-08T12:38:13.758406+0000","last_fullsized":"2023-07-08T12:38:13.758406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:21.663219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759864+0000","last_change":"2023-07-08T12:38:08.561895+0000","last_active":"2023-07-08T12:38:13.759864+0000","last_peered":"2023-07-08T12:38:13.759864+0000","last_clean":"2023-07-08T12:38:13.759864+0000","last_became_active":"2023-07-08T12:38:08.561428+0000","last_became_peered":"2023-07-08T12:38:08.561428+0000","last_unstale":"2023-07-08T12:38:13.759864+0000","last_undegraded":"2023-07-08T12:38:13.759864+0000","last_fullsized":"2023-07-08T12:38:13.759864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:07.081181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605310+0000","last_change":"2023-07-08T12:38:12.592292+0000","last_active":"2023-07-08T12:38:13.605310+0000","last_peered":"2023-07-08T12:38:13.605310+0000","last_clean":"2023-07-08T12:38:13.605310+0000","last_became_active":"2023-07-08T12:38:12.591964+0000","last_became_peered":"2023-07-08T12:38:12.591964+0000","last_unstale":"2023-07-08T12:38:13.605310+0000","last_undegraded":"2023-07-08T12:38:13.605310+0000","last_fullsized":"2023-07-08T12:38:13.605310+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:53.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758158+0000","last_change":"2023-07-08T12:38:07.543683+0000","last_active":"2023-07-08T12:38:13.758158+0000","last_peered":"2023-07-08T12:38:13.758158+0000","last_clean":"2023-07-08T12:38:13.758158+0000","last_became_active":"2023-07-08T12:38:07.543416+0000","last_became_peered":"2023-07-08T12:38:07.543416+0000","last_unstale":"2023-07-08T12:38:13.758158+0000","last_undegraded":"2023-07-08T12:38:13.758158+0000","last_fullsized":"2023-07-08T12:38:13.758158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:56:16.609129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758366+0000","last_change":"2023-07-08T12:38:08.556977+0000","last_active":"2023-07-08T12:38:13.758366+0000","last_peered":"2023-07-08T12:38:13.758366+0000","last_clean":"2023-07-08T12:38:13.758366+0000","last_became_active":"2023-07-08T12:38:08.555976+0000","last_became_peered":"2023-07-08T12:38:08.555976+0000","last_unstale":"2023-07-08T12:38:13.758366+0000","last_undegraded":"2023-07-08T12:38:13.758366+0000","last_fullsized":"2023-07-08T12:38:13.758366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:36.419008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759039+0000","last_change":"2023-07-08T12:38:12.582526+0000","last_active":"2023-07-08T12:38:13.759039+0000","last_peered":"2023-07-08T12:38:13.759039+0000","last_clean":"2023-07-08T12:38:13.759039+0000","last_became_active":"2023-07-08T12:38:12.582130+0000","last_became_peered":"2023-07-08T12:38:12.582130+0000","last_unstale":"2023-07-08T12:38:13.759039+0000","last_undegraded":"2023-07-08T12:38:13.759039+0000","last_fullsized":"2023-07-08T12:38:13.759039+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:31:58.033845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759843+0000","last_change":"2023-07-08T12:38:07.543929+0000","last_active":"2023-07-08T12:38:13.759843+0000","last_peered":"2023-07-08T12:38:13.759843+0000","last_clean":"2023-07-08T12:38:13.759843+0000","last_became_active":"2023-07-08T12:38:07.543693+0000","last_became_peered":"2023-07-08T12:38:07.543693+0000","last_unstale":"2023-07-08T12:38:13.759843+0000","last_undegraded":"2023-07-08T12:38:13.759843+0000","last_fullsized":"2023-07-08T12:38:13.759843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:55:49.487719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604436+0000","last_change":"2023-07-08T12:38:08.548856+0000","last_active":"2023-07-08T12:38:13.604436+0000","last_peered":"2023-07-08T12:38:13.604436+0000","last_clean":"2023-07-08T12:38:13.604436+0000","last_became_active":"2023-07-08T12:38:08.548294+0000","last_became_peered":"2023-07-08T12:38:08.548294+0000","last_unstale":"2023-07-08T12:38:13.604436+0000","last_undegraded":"2023-07-08T12:38:13.604436+0000","last_fullsized":"2023-07-08T12:38:13.604436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:53.873880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.603862+0000","last_change":"2023-07-08T12:38:12.587147+0000","last_active":"2023-07-08T12:38:13.603862+0000","last_peered":"2023-07-08T12:38:13.603862+0000","last_clean":"2023-07-08T12:38:13.603862+0000","last_became_active":"2023-07-08T12:38:12.586709+0000","last_became_peered":"2023-07-08T12:38:12.586709+0000","last_unstale":"2023-07-08T12:38:13.603862+0000","last_undegraded":"2023-07-08T12:38:13.603862+0000","last_fullsized":"2023-07-08T12:38:13.603862+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T12:58:56.632605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759614+0000","last_change":"2023-07-08T12:38:07.545521+0000","last_active":"2023-07-08T12:38:13.759614+0000","last_peered":"2023-07-08T12:38:13.759614+0000","last_clean":"2023-07-08T12:38:13.759614+0000","last_became_active":"2023-07-08T12:38:07.545260+0000","last_became_peered":"2023-07-08T12:38:07.545260+0000","last_unstale":"2023-07-08T12:38:13.759614+0000","last_undegraded":"2023-07-08T12:38:13.759614+0000","last_fullsized":"2023-07-08T12:38:13.759614+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:02.385428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605419+0000","last_change":"2023-07-08T12:38:08.552222+0000","last_active":"2023-07-08T12:38:13.605419+0000","last_peered":"2023-07-08T12:38:13.605419+0000","last_clean":"2023-07-08T12:38:13.605419+0000","last_became_active":"2023-07-08T12:38:08.551940+0000","last_became_peered":"2023-07-08T12:38:08.551940+0000","last_unstale":"2023-07-08T12:38:13.605419+0000","last_undegraded":"2023-07-08T12:38:13.605419+0000","last_fullsized":"2023-07-08T12:38:13.605419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:31:34.141462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604126+0000","last_change":"2023-07-08T12:38:12.589202+0000","last_active":"2023-07-08T12:38:13.604126+0000","last_peered":"2023-07-08T12:38:13.604126+0000","last_clean":"2023-07-08T12:38:13.604126+0000","last_became_active":"2023-07-08T12:38:12.586775+0000","last_became_peered":"2023-07-08T12:38:12.586775+0000","last_unstale":"2023-07-08T12:38:13.604126+0000","last_undegraded":"2023-07-08T12:38:13.604126+0000","last_fullsized":"2023-07-08T12:38:13.604126+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:17:35.606731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758901+0000","last_change":"2023-07-08T12:38:07.542576+0000","last_active":"2023-07-08T12:38:13.758901+0000","last_peered":"2023-07-08T12:38:13.758901+0000","last_clean":"2023-07-08T12:38:13.758901+0000","last_became_active":"2023-07-08T12:38:07.542296+0000","last_became_peered":"2023-07-08T12:38:07.542296+0000","last_unstale":"2023-07-08T12:38:13.758901+0000","last_undegraded":"2023-07-08T12:38:13.758901+0000","last_fullsized":"2023-07-08T12:38:13.758901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:28:48.361016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758718+0000","last_change":"2023-07-08T12:38:08.560794+0000","last_active":"2023-07-08T12:38:13.758718+0000","last_peered":"2023-07-08T12:38:13.758718+0000","last_clean":"2023-07-08T12:38:13.758718+0000","last_became_active":"2023-07-08T12:38:08.560405+0000","last_became_peered":"2023-07-08T12:38:08.560405+0000","last_unstale":"2023-07-08T12:38:13.758718+0000","last_undegraded":"2023-07-08T12:38:13.758718+0000","last_fullsized":"2023-07-08T12:38:13.758718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:52:43.348476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604255+0000","last_change":"2023-07-08T12:38:12.585051+0000","last_active":"2023-07-08T12:38:13.604255+0000","last_peered":"2023-07-08T12:38:13.604255+0000","last_clean":"2023-07-08T12:38:13.604255+0000","last_became_active":"2023-07-08T12:38:12.583082+0000","last_became_peered":"2023-07-08T12:38:12.583082+0000","last_unstale":"2023-07-08T12:38:13.604255+0000","last_undegraded":"2023-07-08T12:38:13.604255+0000","last_fullsized":"2023-07-08T12:38:13.604255+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:02:43.298466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604866+0000","last_change":"2023-07-08T12:38:07.537900+0000","last_active":"2023-07-08T12:38:13.604866+0000","last_peered":"2023-07-08T12:38:13.604866+0000","last_clean":"2023-07-08T12:38:13.604866+0000","last_became_active":"2023-07-08T12:38:07.536896+0000","last_became_peered":"2023-07-08T12:38:07.536896+0000","last_unstale":"2023-07-08T12:38:13.604866+0000","last_undegraded":"2023-07-08T12:38:13.604866+0000","last_fullsized":"2023-07-08T12:38:13.604866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:55.151640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759269+0000","last_change":"2023-07-08T12:38:08.569854+0000","last_active":"2023-07-08T12:38:13.759269+0000","last_peered":"2023-07-08T12:38:13.759269+0000","last_clean":"2023-07-08T12:38:13.759269+0000","last_became_active":"2023-07-08T12:38:08.569549+0000","last_became_peered":"2023-07-08T12:38:08.569549+0000","last_unstale":"2023-07-08T12:38:13.759269+0000","last_undegraded":"2023-07-08T12:38:13.759269+0000","last_fullsized":"2023-07-08T12:38:13.759269+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:15.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604571+0000","last_change":"2023-07-08T12:38:12.580215+0000","last_active":"2023-07-08T12:38:13.604571+0000","last_peered":"2023-07-08T12:38:13.604571+0000","last_clean":"2023-07-08T12:38:13.604571+0000","last_became_active":"2023-07-08T12:38:12.579706+0000","last_became_peered":"2023-07-08T12:38:12.579706+0000","last_unstale":"2023-07-08T12:38:13.604571+0000","last_undegraded":"2023-07-08T12:38:13.604571+0000","last_fullsized":"2023-07-08T12:38:13.604571+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:08:35.080577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758601+0000","last_change":"2023-07-08T12:38:07.542376+0000","last_active":"2023-07-08T12:38:13.758601+0000","last_peered":"2023-07-08T12:38:13.758601+0000","last_clean":"2023-07-08T12:38:13.758601+0000","last_became_active":"2023-07-08T12:38:07.542046+0000","last_became_peered":"2023-07-08T12:38:07.542046+0000","last_unstale":"2023-07-08T12:38:13.758601+0000","last_undegraded":"2023-07-08T12:38:13.758601+0000","last_fullsized":"2023-07-08T12:38:13.758601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:28:48.361016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605011+0000","last_change":"2023-07-08T12:38:08.553132+0000","last_active":"2023-07-08T12:38:13.605011+0000","last_peered":"2023-07-08T12:38:13.605011+0000","last_clean":"2023-07-08T12:38:13.605011+0000","last_became_active":"2023-07-08T12:38:08.552711+0000","last_became_peered":"2023-07-08T12:38:08.552711+0000","last_unstale":"2023-07-08T12:38:13.605011+0000","last_undegraded":"2023-07-08T12:38:13.605011+0000","last_fullsized":"2023-07-08T12:38:13.605011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T16:24:56.036654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759123+0000","last_change":"2023-07-08T12:38:12.597633+0000","last_active":"2023-07-08T12:38:13.759123+0000","last_peered":"2023-07-08T12:38:13.759123+0000","last_clean":"2023-07-08T12:38:13.759123+0000","last_became_active":"2023-07-08T12:38:12.597211+0000","last_became_peered":"2023-07-08T12:38:12.597211+0000","last_unstale":"2023-07-08T12:38:13.759123+0000","last_undegraded":"2023-07-08T12:38:13.759123+0000","last_fullsized":"2023-07-08T12:38:13.759123+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:04:47.003361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758887+0000","last_change":"2023-07-08T12:38:07.542040+0000","last_active":"2023-07-08T12:38:13.758887+0000","last_peered":"2023-07-08T12:38:13.758887+0000","last_clean":"2023-07-08T12:38:13.758887+0000","last_became_active":"2023-07-08T12:38:07.541743+0000","last_became_peered":"2023-07-08T12:38:07.541743+0000","last_unstale":"2023-07-08T12:38:13.758887+0000","last_undegraded":"2023-07-08T12:38:13.758887+0000","last_fullsized":"2023-07-08T12:38:13.758887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:41:49.220614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606520+0000","last_change":"2023-07-08T12:38:08.548025+0000","last_active":"2023-07-08T12:38:13.606520+0000","last_peered":"2023-07-08T12:38:13.606520+0000","last_clean":"2023-07-08T12:38:13.606520+0000","last_became_active":"2023-07-08T12:38:08.547723+0000","last_became_peered":"2023-07-08T12:38:08.547723+0000","last_unstale":"2023-07-08T12:38:13.606520+0000","last_undegraded":"2023-07-08T12:38:13.606520+0000","last_fullsized":"2023-07-08T12:38:13.606520+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:01:21.861852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605658+0000","last_change":"2023-07-08T12:38:12.587540+0000","last_active":"2023-07-08T12:38:13.605658+0000","last_peered":"2023-07-08T12:38:13.605658+0000","last_clean":"2023-07-08T12:38:13.605658+0000","last_became_active":"2023-07-08T12:38:12.587180+0000","last_became_peered":"2023-07-08T12:38:12.587180+0000","last_unstale":"2023-07-08T12:38:13.605658+0000","last_undegraded":"2023-07-08T12:38:13.605658+0000","last_fullsized":"2023-07-08T12:38:13.605658+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:31:58.033845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605007+0000","last_change":"2023-07-08T12:38:07.535635+0000","last_active":"2023-07-08T12:38:13.605007+0000","last_peered":"2023-07-08T12:38:13.605007+0000","last_clean":"2023-07-08T12:38:13.605007+0000","last_became_active":"2023-07-08T12:38:07.535335+0000","last_became_peered":"2023-07-08T12:38:07.535335+0000","last_unstale":"2023-07-08T12:38:13.605007+0000","last_undegraded":"2023-07-08T12:38:13.605007+0000","last_fullsized":"2023-07-08T12:38:13.605007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:15:42.329406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606965+0000","last_change":"2023-07-08T12:38:08.553024+0000","last_active":"2023-07-08T12:38:13.606965+0000","last_peered":"2023-07-08T12:38:13.606965+0000","last_clean":"2023-07-08T12:38:13.606965+0000","last_became_active":"2023-07-08T12:38:08.552696+0000","last_became_peered":"2023-07-08T12:38:08.552696+0000","last_unstale":"2023-07-08T12:38:13.606965+0000","last_undegraded":"2023-07-08T12:38:13.606965+0000","last_fullsized":"2023-07-08T12:38:13.606965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:33:05.815083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606715+0000","last_change":"2023-07-08T12:38:12.584493+0000","last_active":"2023-07-08T12:38:13.606715+0000","last_peered":"2023-07-08T12:38:13.606715+0000","last_clean":"2023-07-08T12:38:13.606715+0000","last_became_active":"2023-07-08T12:38:12.584087+0000","last_became_peered":"2023-07-08T12:38:12.584087+0000","last_unstale":"2023-07-08T12:38:13.606715+0000","last_undegraded":"2023-07-08T12:38:13.606715+0000","last_fullsized":"2023-07-08T12:38:13.606715+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:10:54.808570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758207+0000","last_change":"2023-07-08T12:38:07.540092+0000","last_active":"2023-07-08T12:38:13.758207+0000","last_peered":"2023-07-08T12:38:13.758207+0000","last_clean":"2023-07-08T12:38:13.758207+0000","last_became_active":"2023-07-08T12:38:07.539789+0000","last_became_peered":"2023-07-08T12:38:07.539789+0000","last_unstale":"2023-07-08T12:38:13.758207+0000","last_undegraded":"2023-07-08T12:38:13.758207+0000","last_fullsized":"2023-07-08T12:38:13.758207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:39.629914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759601+0000","last_change":"2023-07-08T12:38:08.564633+0000","last_active":"2023-07-08T12:38:13.759601+0000","last_peered":"2023-07-08T12:38:13.759601+0000","last_clean":"2023-07-08T12:38:13.759601+0000","last_became_active":"2023-07-08T12:38:08.564323+0000","last_became_peered":"2023-07-08T12:38:08.564323+0000","last_unstale":"2023-07-08T12:38:13.759601+0000","last_undegraded":"2023-07-08T12:38:13.759601+0000","last_fullsized":"2023-07-08T12:38:13.759601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:28.004982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759180+0000","last_change":"2023-07-08T12:38:12.586250+0000","last_active":"2023-07-08T12:38:13.759180+0000","last_peered":"2023-07-08T12:38:13.759180+0000","last_clean":"2023-07-08T12:38:13.759180+0000","last_became_active":"2023-07-08T12:38:12.585838+0000","last_became_peered":"2023-07-08T12:38:12.585838+0000","last_unstale":"2023-07-08T12:38:13.759180+0000","last_undegraded":"2023-07-08T12:38:13.759180+0000","last_fullsized":"2023-07-08T12:38:13.759180+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:06:30.594880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759317+0000","last_change":"2023-07-08T12:38:03.160177+0000","last_active":"2023-07-08T12:38:13.759317+0000","last_peered":"2023-07-08T12:38:13.759317+0000","last_clean":"2023-07-08T12:38:13.759317+0000","last_became_active":"2023-07-08T12:38:03.159864+0000","last_became_peered":"2023-07-08T12:38:03.159864+0000","last_unstale":"2023-07-08T12:38:13.759317+0000","last_undegraded":"2023-07-08T12:38:13.759317+0000","last_fullsized":"2023-07-08T12:38:13.759317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:31:56.449203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604812+0000","last_change":"2023-07-08T12:38:07.537328+0000","last_active":"2023-07-08T12:38:13.604812+0000","last_peered":"2023-07-08T12:38:13.604812+0000","last_clean":"2023-07-08T12:38:13.604812+0000","last_became_active":"2023-07-08T12:38:07.537038+0000","last_became_peered":"2023-07-08T12:38:07.537038+0000","last_unstale":"2023-07-08T12:38:13.604812+0000","last_undegraded":"2023-07-08T12:38:13.604812+0000","last_fullsized":"2023-07-08T12:38:13.604812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:39.629914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760774+0000","last_change":"2023-07-08T12:38:08.558271+0000","last_active":"2023-07-08T12:38:13.760774+0000","last_peered":"2023-07-08T12:38:13.760774+0000","last_clean":"2023-07-08T12:38:13.760774+0000","last_became_active":"2023-07-08T12:38:08.557598+0000","last_became_peered":"2023-07-08T12:38:08.557598+0000","last_unstale":"2023-07-08T12:38:13.760774+0000","last_undegraded":"2023-07-08T12:38:13.760774+0000","last_fullsized":"2023-07-08T12:38:13.760774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:49.502059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759252+0000","last_change":"2023-07-08T12:38:12.591995+0000","last_active":"2023-07-08T12:38:13.759252+0000","last_peered":"2023-07-08T12:38:13.759252+0000","last_clean":"2023-07-08T12:38:13.759252+0000","last_became_active":"2023-07-08T12:38:12.591624+0000","last_became_peered":"2023-07-08T12:38:12.591624+0000","last_unstale":"2023-07-08T12:38:13.759252+0000","last_undegraded":"2023-07-08T12:38:13.759252+0000","last_fullsized":"2023-07-08T12:38:13.759252+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:57.916979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605001+0000","last_change":"2023-07-08T12:38:02.384639+0000","last_active":"2023-07-08T12:38:13.605001+0000","last_peered":"2023-07-08T12:38:13.605001+0000","last_clean":"2023-07-08T12:38:13.605001+0000","last_became_active":"2023-07-08T12:38:02.384372+0000","last_became_peered":"2023-07-08T12:38:02.384372+0000","last_unstale":"2023-07-08T12:38:13.605001+0000","last_undegraded":"2023-07-08T12:38:13.605001+0000","last_fullsized":"2023-07-08T12:38:13.605001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:29.599162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605155+0000","last_change":"2023-07-08T12:38:07.541393+0000","last_active":"2023-07-08T12:38:13.605155+0000","last_peered":"2023-07-08T12:38:13.605155+0000","last_clean":"2023-07-08T12:38:13.605155+0000","last_became_active":"2023-07-08T12:38:07.541155+0000","last_became_peered":"2023-07-08T12:38:07.541155+0000","last_unstale":"2023-07-08T12:38:13.605155+0000","last_undegraded":"2023-07-08T12:38:13.605155+0000","last_fullsized":"2023-07-08T12:38:13.605155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:51:13.319094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759210+0000","last_change":"2023-07-08T12:38:08.550498+0000","last_active":"2023-07-08T12:38:13.759210+0000","last_peered":"2023-07-08T12:38:13.759210+0000","last_clean":"2023-07-08T12:38:13.759210+0000","last_became_active":"2023-07-08T12:38:08.550106+0000","last_became_peered":"2023-07-08T12:38:08.550106+0000","last_unstale":"2023-07-08T12:38:13.759210+0000","last_undegraded":"2023-07-08T12:38:13.759210+0000","last_fullsized":"2023-07-08T12:38:13.759210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:18:27.333138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604527+0000","last_change":"2023-07-08T12:38:12.586451+0000","last_active":"2023-07-08T12:38:13.604527+0000","last_peered":"2023-07-08T12:38:13.604527+0000","last_clean":"2023-07-08T12:38:13.604527+0000","last_became_active":"2023-07-08T12:38:12.582823+0000","last_became_peered":"2023-07-08T12:38:12.582823+0000","last_unstale":"2023-07-08T12:38:13.604527+0000","last_undegraded":"2023-07-08T12:38:13.604527+0000","last_fullsized":"2023-07-08T12:38:13.604527+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:18:34.491455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604236+0000","last_change":"2023-07-08T12:38:03.212466+0000","last_active":"2023-07-08T12:38:13.604236+0000","last_peered":"2023-07-08T12:38:13.604236+0000","last_clean":"2023-07-08T12:38:13.604236+0000","last_became_active":"2023-07-08T12:38:03.212182+0000","last_became_peered":"2023-07-08T12:38:03.212182+0000","last_unstale":"2023-07-08T12:38:13.604236+0000","last_undegraded":"2023-07-08T12:38:13.604236+0000","last_fullsized":"2023-07-08T12:38:13.604236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:06:34.067771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758091+0000","last_change":"2023-07-08T12:38:07.538415+0000","last_active":"2023-07-08T12:38:13.758091+0000","last_peered":"2023-07-08T12:38:13.758091+0000","last_clean":"2023-07-08T12:38:13.758091+0000","last_became_active":"2023-07-08T12:38:07.537871+0000","last_became_peered":"2023-07-08T12:38:07.537871+0000","last_unstale":"2023-07-08T12:38:13.758091+0000","last_undegraded":"2023-07-08T12:38:13.758091+0000","last_fullsized":"2023-07-08T12:38:13.758091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:00:59.245798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759478+0000","last_change":"2023-07-08T12:38:08.554865+0000","last_active":"2023-07-08T12:38:13.759478+0000","last_peered":"2023-07-08T12:38:13.759478+0000","last_clean":"2023-07-08T12:38:13.759478+0000","last_became_active":"2023-07-08T12:38:08.554591+0000","last_became_peered":"2023-07-08T12:38:08.554591+0000","last_unstale":"2023-07-08T12:38:13.759478+0000","last_undegraded":"2023-07-08T12:38:13.759478+0000","last_fullsized":"2023-07-08T12:38:13.759478+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:53.873880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758470+0000","last_change":"2023-07-08T12:38:12.579919+0000","last_active":"2023-07-08T12:38:13.758470+0000","last_peered":"2023-07-08T12:38:13.758470+0000","last_clean":"2023-07-08T12:38:13.758470+0000","last_became_active":"2023-07-08T12:38:12.579500+0000","last_became_peered":"2023-07-08T12:38:12.579500+0000","last_unstale":"2023-07-08T12:38:13.758470+0000","last_undegraded":"2023-07-08T12:38:13.758470+0000","last_fullsized":"2023-07-08T12:38:13.758470+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:53.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605571+0000","last_change":"2023-07-08T12:38:02.380175+0000","last_active":"2023-07-08T12:38:13.605571+0000","last_peered":"2023-07-08T12:38:13.605571+0000","last_clean":"2023-07-08T12:38:13.605571+0000","last_became_active":"2023-07-08T12:38:02.379867+0000","last_became_peered":"2023-07-08T12:38:02.379867+0000","last_unstale":"2023-07-08T12:38:13.605571+0000","last_undegraded":"2023-07-08T12:38:13.605571+0000","last_fullsized":"2023-07-08T12:38:13.605571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:35:57.492852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758117+0000","last_change":"2023-07-08T12:37:58.222371+0000","last_active":"2023-07-08T12:38:13.758117+0000","last_peered":"2023-07-08T12:38:13.758117+0000","last_clean":"2023-07-08T12:38:13.758117+0000","last_became_active":"2023-07-08T12:37:58.221381+0000","last_became_peered":"2023-07-08T12:37:58.221381+0000","last_unstale":"2023-07-08T12:38:13.758117+0000","last_undegraded":"2023-07-08T12:38:13.758117+0000","last_fullsized":"2023-07-08T12:38:13.758117+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:37:56.327863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:37:56.327863+0000","last_clean_scrub_stamp":"2023-07-08T12:37:56.327863+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:44.968733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758891+0000","last_change":"2023-07-08T12:38:07.536547+0000","last_active":"2023-07-08T12:38:13.758891+0000","last_peered":"2023-07-08T12:38:13.758891+0000","last_clean":"2023-07-08T12:38:13.758891+0000","last_became_active":"2023-07-08T12:38:07.536255+0000","last_became_peered":"2023-07-08T12:38:07.536255+0000","last_unstale":"2023-07-08T12:38:13.758891+0000","last_undegraded":"2023-07-08T12:38:13.758891+0000","last_fullsized":"2023-07-08T12:38:13.758891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:41:53.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759285+0000","last_change":"2023-07-08T12:38:08.550673+0000","last_active":"2023-07-08T12:38:13.759285+0000","last_peered":"2023-07-08T12:38:13.759285+0000","last_clean":"2023-07-08T12:38:13.759285+0000","last_became_active":"2023-07-08T12:38:08.550347+0000","last_became_peered":"2023-07-08T12:38:08.550347+0000","last_unstale":"2023-07-08T12:38:13.759285+0000","last_undegraded":"2023-07-08T12:38:13.759285+0000","last_fullsized":"2023-07-08T12:38:13.759285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:03:34.056111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605442+0000","last_change":"2023-07-08T12:38:12.597251+0000","last_active":"2023-07-08T12:38:13.605442+0000","last_peered":"2023-07-08T12:38:13.605442+0000","last_clean":"2023-07-08T12:38:13.605442+0000","last_became_active":"2023-07-08T12:38:12.596856+0000","last_became_peered":"2023-07-08T12:38:12.596856+0000","last_unstale":"2023-07-08T12:38:13.605442+0000","last_undegraded":"2023-07-08T12:38:13.605442+0000","last_fullsized":"2023-07-08T12:38:13.605442+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:11.124280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758299+0000","last_change":"2023-07-08T12:38:02.385193+0000","last_active":"2023-07-08T12:38:13.758299+0000","last_peered":"2023-07-08T12:38:13.758299+0000","last_clean":"2023-07-08T12:38:13.758299+0000","last_became_active":"2023-07-08T12:38:02.384865+0000","last_became_peered":"2023-07-08T12:38:02.384865+0000","last_unstale":"2023-07-08T12:38:13.758299+0000","last_undegraded":"2023-07-08T12:38:13.758299+0000","last_fullsized":"2023-07-08T12:38:13.758299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:30.260044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.757707+0000","last_change":"2023-07-08T12:38:07.543452+0000","last_active":"2023-07-08T12:38:13.757707+0000","last_peered":"2023-07-08T12:38:13.757707+0000","last_clean":"2023-07-08T12:38:13.757707+0000","last_became_active":"2023-07-08T12:38:07.543141+0000","last_became_peered":"2023-07-08T12:38:07.543141+0000","last_unstale":"2023-07-08T12:38:13.757707+0000","last_undegraded":"2023-07-08T12:38:13.757707+0000","last_fullsized":"2023-07-08T12:38:13.757707+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:03:16.870782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759081+0000","last_change":"2023-07-08T12:38:08.570468+0000","last_active":"2023-07-08T12:38:13.759081+0000","last_peered":"2023-07-08T12:38:13.759081+0000","last_clean":"2023-07-08T12:38:13.759081+0000","last_became_active":"2023-07-08T12:38:08.570183+0000","last_became_peered":"2023-07-08T12:38:08.570183+0000","last_unstale":"2023-07-08T12:38:13.759081+0000","last_undegraded":"2023-07-08T12:38:13.759081+0000","last_fullsized":"2023-07-08T12:38:13.759081+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:32:02.608167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759172+0000","last_change":"2023-07-08T12:38:12.597186+0000","last_active":"2023-07-08T12:38:13.759172+0000","last_peered":"2023-07-08T12:38:13.759172+0000","last_clean":"2023-07-08T12:38:13.759172+0000","last_became_active":"2023-07-08T12:38:12.596772+0000","last_became_peered":"2023-07-08T12:38:12.596772+0000","last_unstale":"2023-07-08T12:38:13.759172+0000","last_undegraded":"2023-07-08T12:38:13.759172+0000","last_fullsized":"2023-07-08T12:38:13.759172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:06:56.291254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758758+0000","last_change":"2023-07-08T12:38:03.160344+0000","last_active":"2023-07-08T12:38:13.758758+0000","last_peered":"2023-07-08T12:38:13.758758+0000","last_clean":"2023-07-08T12:38:13.758758+0000","last_became_active":"2023-07-08T12:38:03.160009+0000","last_became_peered":"2023-07-08T12:38:03.160009+0000","last_unstale":"2023-07-08T12:38:13.758758+0000","last_undegraded":"2023-07-08T12:38:13.758758+0000","last_fullsized":"2023-07-08T12:38:13.758758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:35:57.492852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759529+0000","last_change":"2023-07-08T12:38:07.539629+0000","last_active":"2023-07-08T12:38:13.759529+0000","last_peered":"2023-07-08T12:38:13.759529+0000","last_clean":"2023-07-08T12:38:13.759529+0000","last_became_active":"2023-07-08T12:38:07.537615+0000","last_became_peered":"2023-07-08T12:38:07.537615+0000","last_unstale":"2023-07-08T12:38:13.759529+0000","last_undegraded":"2023-07-08T12:38:13.759529+0000","last_fullsized":"2023-07-08T12:38:13.759529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:55.151640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759477+0000","last_change":"2023-07-08T12:38:08.556275+0000","last_active":"2023-07-08T12:38:13.759477+0000","last_peered":"2023-07-08T12:38:13.759477+0000","last_clean":"2023-07-08T12:38:13.759477+0000","last_became_active":"2023-07-08T12:38:08.556016+0000","last_became_peered":"2023-07-08T12:38:08.556016+0000","last_unstale":"2023-07-08T12:38:13.759477+0000","last_undegraded":"2023-07-08T12:38:13.759477+0000","last_fullsized":"2023-07-08T12:38:13.759477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605611+0000","last_change":"2023-07-08T12:38:12.584715+0000","last_active":"2023-07-08T12:38:13.605611+0000","last_peered":"2023-07-08T12:38:13.605611+0000","last_clean":"2023-07-08T12:38:13.605611+0000","last_became_active":"2023-07-08T12:38:12.584249+0000","last_became_peered":"2023-07-08T12:38:12.584249+0000","last_unstale":"2023-07-08T12:38:13.605611+0000","last_undegraded":"2023-07-08T12:38:13.605611+0000","last_fullsized":"2023-07-08T12:38:13.605611+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:34:52.829894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759931+0000","last_change":"2023-07-08T12:38:07.537430+0000","last_active":"2023-07-08T12:38:13.759931+0000","last_peered":"2023-07-08T12:38:13.759931+0000","last_clean":"2023-07-08T12:38:13.759931+0000","last_became_active":"2023-07-08T12:38:07.537043+0000","last_became_peered":"2023-07-08T12:38:07.537043+0000","last_unstale":"2023-07-08T12:38:13.759931+0000","last_undegraded":"2023-07-08T12:38:13.759931+0000","last_fullsized":"2023-07-08T12:38:13.759931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:18:17.308237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760054+0000","last_change":"2023-07-08T12:38:02.385821+0000","last_active":"2023-07-08T12:38:13.760054+0000","last_peered":"2023-07-08T12:38:13.760054+0000","last_clean":"2023-07-08T12:38:13.760054+0000","last_became_active":"2023-07-08T12:38:02.385512+0000","last_became_peered":"2023-07-08T12:38:02.385512+0000","last_unstale":"2023-07-08T12:38:13.760054+0000","last_undegraded":"2023-07-08T12:38:13.760054+0000","last_fullsized":"2023-07-08T12:38:13.760054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:49.796115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758149+0000","last_change":"2023-07-08T12:38:08.556555+0000","last_active":"2023-07-08T12:38:13.758149+0000","last_peered":"2023-07-08T12:38:13.758149+0000","last_clean":"2023-07-08T12:38:13.758149+0000","last_became_active":"2023-07-08T12:38:08.555822+0000","last_became_peered":"2023-07-08T12:38:08.555822+0000","last_unstale":"2023-07-08T12:38:13.758149+0000","last_undegraded":"2023-07-08T12:38:13.758149+0000","last_fullsized":"2023-07-08T12:38:13.758149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:15.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758115+0000","last_change":"2023-07-08T12:38:12.587833+0000","last_active":"2023-07-08T12:38:13.758115+0000","last_peered":"2023-07-08T12:38:13.758115+0000","last_clean":"2023-07-08T12:38:13.758115+0000","last_became_active":"2023-07-08T12:38:12.587465+0000","last_became_peered":"2023-07-08T12:38:12.587465+0000","last_unstale":"2023-07-08T12:38:13.758115+0000","last_undegraded":"2023-07-08T12:38:13.758115+0000","last_fullsized":"2023-07-08T12:38:13.758115+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:32.048081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759111+0000","last_change":"2023-07-08T12:38:02.385388+0000","last_active":"2023-07-08T12:38:13.759111+0000","last_peered":"2023-07-08T12:38:13.759111+0000","last_clean":"2023-07-08T12:38:13.759111+0000","last_became_active":"2023-07-08T12:38:02.385110+0000","last_became_peered":"2023-07-08T12:38:02.385110+0000","last_unstale":"2023-07-08T12:38:13.759111+0000","last_undegraded":"2023-07-08T12:38:13.759111+0000","last_fullsized":"2023-07-08T12:38:13.759111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:52:02.887135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759215+0000","last_change":"2023-07-08T12:38:07.544210+0000","last_active":"2023-07-08T12:38:13.759215+0000","last_peered":"2023-07-08T12:38:13.759215+0000","last_clean":"2023-07-08T12:38:13.759215+0000","last_became_active":"2023-07-08T12:38:07.543870+0000","last_became_peered":"2023-07-08T12:38:07.543870+0000","last_unstale":"2023-07-08T12:38:13.759215+0000","last_undegraded":"2023-07-08T12:38:13.759215+0000","last_fullsized":"2023-07-08T12:38:13.759215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:27.000382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606243+0000","last_change":"2023-07-08T12:38:08.548933+0000","last_active":"2023-07-08T12:38:13.606243+0000","last_peered":"2023-07-08T12:38:13.606243+0000","last_clean":"2023-07-08T12:38:13.606243+0000","last_became_active":"2023-07-08T12:38:08.548655+0000","last_became_peered":"2023-07-08T12:38:08.548655+0000","last_unstale":"2023-07-08T12:38:13.606243+0000","last_undegraded":"2023-07-08T12:38:13.606243+0000","last_fullsized":"2023-07-08T12:38:13.606243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:33:14.696652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759816+0000","last_change":"2023-07-08T12:38:12.597637+0000","last_active":"2023-07-08T12:38:13.759816+0000","last_peered":"2023-07-08T12:38:13.759816+0000","last_clean":"2023-07-08T12:38:13.759816+0000","last_became_active":"2023-07-08T12:38:12.597184+0000","last_became_peered":"2023-07-08T12:38:12.597184+0000","last_unstale":"2023-07-08T12:38:13.759816+0000","last_undegraded":"2023-07-08T12:38:13.759816+0000","last_fullsized":"2023-07-08T12:38:13.759816+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:44.677613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759341+0000","last_change":"2023-07-08T12:38:07.546269+0000","last_active":"2023-07-08T12:38:13.759341+0000","last_peered":"2023-07-08T12:38:13.759341+0000","last_clean":"2023-07-08T12:38:13.759341+0000","last_became_active":"2023-07-08T12:38:07.545942+0000","last_became_peered":"2023-07-08T12:38:07.545942+0000","last_unstale":"2023-07-08T12:38:13.759341+0000","last_undegraded":"2023-07-08T12:38:13.759341+0000","last_fullsized":"2023-07-08T12:38:13.759341+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:55.151640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604575+0000","last_change":"2023-07-08T12:38:08.566129+0000","last_active":"2023-07-08T12:38:13.604575+0000","last_peered":"2023-07-08T12:38:13.604575+0000","last_clean":"2023-07-08T12:38:13.604575+0000","last_became_active":"2023-07-08T12:38:08.565780+0000","last_became_peered":"2023-07-08T12:38:08.565780+0000","last_unstale":"2023-07-08T12:38:13.604575+0000","last_undegraded":"2023-07-08T12:38:13.604575+0000","last_fullsized":"2023-07-08T12:38:13.604575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:52:43.348476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758357+0000","last_change":"2023-07-08T12:38:12.582590+0000","last_active":"2023-07-08T12:38:13.758357+0000","last_peered":"2023-07-08T12:38:13.758357+0000","last_clean":"2023-07-08T12:38:13.758357+0000","last_became_active":"2023-07-08T12:38:12.582179+0000","last_became_peered":"2023-07-08T12:38:12.582179+0000","last_unstale":"2023-07-08T12:38:13.758357+0000","last_undegraded":"2023-07-08T12:38:13.758357+0000","last_fullsized":"2023-07-08T12:38:13.758357+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:39.801225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604721+0000","last_change":"2023-07-08T12:38:07.544462+0000","last_active":"2023-07-08T12:38:13.604721+0000","last_peered":"2023-07-08T12:38:13.604721+0000","last_clean":"2023-07-08T12:38:13.604721+0000","last_became_active":"2023-07-08T12:38:07.544171+0000","last_became_peered":"2023-07-08T12:38:07.544171+0000","last_unstale":"2023-07-08T12:38:13.604721+0000","last_undegraded":"2023-07-08T12:38:13.604721+0000","last_fullsized":"2023-07-08T12:38:13.604721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:14.285892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605091+0000","last_change":"2023-07-08T12:38:08.557546+0000","last_active":"2023-07-08T12:38:13.605091+0000","last_peered":"2023-07-08T12:38:13.605091+0000","last_clean":"2023-07-08T12:38:13.605091+0000","last_became_active":"2023-07-08T12:38:08.557262+0000","last_became_peered":"2023-07-08T12:38:08.557262+0000","last_unstale":"2023-07-08T12:38:13.605091+0000","last_undegraded":"2023-07-08T12:38:13.605091+0000","last_fullsized":"2023-07-08T12:38:13.605091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:11:12.440266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605361+0000","last_change":"2023-07-08T12:38:12.597319+0000","last_active":"2023-07-08T12:38:13.605361+0000","last_peered":"2023-07-08T12:38:13.605361+0000","last_clean":"2023-07-08T12:38:13.605361+0000","last_became_active":"2023-07-08T12:38:12.596928+0000","last_became_peered":"2023-07-08T12:38:12.596928+0000","last_unstale":"2023-07-08T12:38:13.605361+0000","last_undegraded":"2023-07-08T12:38:13.605361+0000","last_fullsized":"2023-07-08T12:38:13.605361+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:39.375709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604658+0000","last_change":"2023-07-08T12:38:07.540676+0000","last_active":"2023-07-08T12:38:13.604658+0000","last_peered":"2023-07-08T12:38:13.604658+0000","last_clean":"2023-07-08T12:38:13.604658+0000","last_became_active":"2023-07-08T12:38:07.540389+0000","last_became_peered":"2023-07-08T12:38:07.540389+0000","last_unstale":"2023-07-08T12:38:13.604658+0000","last_undegraded":"2023-07-08T12:38:13.604658+0000","last_fullsized":"2023-07-08T12:38:13.604658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:27.000382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606089+0000","last_change":"2023-07-08T12:38:08.546186+0000","last_active":"2023-07-08T12:38:13.606089+0000","last_peered":"2023-07-08T12:38:13.606089+0000","last_clean":"2023-07-08T12:38:13.606089+0000","last_became_active":"2023-07-08T12:38:08.545902+0000","last_became_peered":"2023-07-08T12:38:08.545902+0000","last_unstale":"2023-07-08T12:38:13.606089+0000","last_undegraded":"2023-07-08T12:38:13.606089+0000","last_fullsized":"2023-07-08T12:38:13.606089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:37:20.499952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604484+0000","last_change":"2023-07-08T12:38:12.581579+0000","last_active":"2023-07-08T12:38:13.604484+0000","last_peered":"2023-07-08T12:38:13.604484+0000","last_clean":"2023-07-08T12:38:13.604484+0000","last_became_active":"2023-07-08T12:38:12.581118+0000","last_became_peered":"2023-07-08T12:38:12.581118+0000","last_unstale":"2023-07-08T12:38:13.604484+0000","last_undegraded":"2023-07-08T12:38:13.604484+0000","last_fullsized":"2023-07-08T12:38:13.604484+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:48:23.748317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758339+0000","last_change":"2023-07-08T12:38:07.541066+0000","last_active":"2023-07-08T12:38:13.758339+0000","last_peered":"2023-07-08T12:38:13.758339+0000","last_clean":"2023-07-08T12:38:13.758339+0000","last_became_active":"2023-07-08T12:38:07.540873+0000","last_became_peered":"2023-07-08T12:38:07.540873+0000","last_unstale":"2023-07-08T12:38:13.758339+0000","last_undegraded":"2023-07-08T12:38:13.758339+0000","last_fullsized":"2023-07-08T12:38:13.758339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:34.328010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758518+0000","last_change":"2023-07-08T12:38:08.553600+0000","last_active":"2023-07-08T12:38:13.758518+0000","last_peered":"2023-07-08T12:38:13.758518+0000","last_clean":"2023-07-08T12:38:13.758518+0000","last_became_active":"2023-07-08T12:38:08.552418+0000","last_became_peered":"2023-07-08T12:38:08.552418+0000","last_unstale":"2023-07-08T12:38:13.758518+0000","last_undegraded":"2023-07-08T12:38:13.758518+0000","last_fullsized":"2023-07-08T12:38:13.758518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:36:03.651816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758948+0000","last_change":"2023-07-08T12:38:12.590123+0000","last_active":"2023-07-08T12:38:13.758948+0000","last_peered":"2023-07-08T12:38:13.758948+0000","last_clean":"2023-07-08T12:38:13.758948+0000","last_became_active":"2023-07-08T12:38:12.589586+0000","last_became_peered":"2023-07-08T12:38:12.589586+0000","last_unstale":"2023-07-08T12:38:13.758948+0000","last_undegraded":"2023-07-08T12:38:13.758948+0000","last_fullsized":"2023-07-08T12:38:13.758948+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:03:21.918481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759739+0000","last_change":"2023-07-08T12:38:07.541648+0000","last_active":"2023-07-08T12:38:13.759739+0000","last_peered":"2023-07-08T12:38:13.759739+0000","last_clean":"2023-07-08T12:38:13.759739+0000","last_became_active":"2023-07-08T12:38:07.541284+0000","last_became_peered":"2023-07-08T12:38:07.541284+0000","last_unstale":"2023-07-08T12:38:13.759739+0000","last_undegraded":"2023-07-08T12:38:13.759739+0000","last_fullsized":"2023-07-08T12:38:13.759739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:36.285500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606128+0000","last_change":"2023-07-08T12:38:08.552907+0000","last_active":"2023-07-08T12:38:13.606128+0000","last_peered":"2023-07-08T12:38:13.606128+0000","last_clean":"2023-07-08T12:38:13.606128+0000","last_became_active":"2023-07-08T12:38:08.552595+0000","last_became_peered":"2023-07-08T12:38:08.552595+0000","last_unstale":"2023-07-08T12:38:13.606128+0000","last_undegraded":"2023-07-08T12:38:13.606128+0000","last_fullsized":"2023-07-08T12:38:13.606128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:02:09.276408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759378+0000","last_change":"2023-07-08T12:38:08.554379+0000","last_active":"2023-07-08T12:38:13.759378+0000","last_peered":"2023-07-08T12:38:13.759378+0000","last_clean":"2023-07-08T12:38:13.759378+0000","last_became_active":"2023-07-08T12:38:08.554092+0000","last_became_peered":"2023-07-08T12:38:08.554092+0000","last_unstale":"2023-07-08T12:38:13.759378+0000","last_undegraded":"2023-07-08T12:38:13.759378+0000","last_fullsized":"2023-07-08T12:38:13.759378+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:36:56.254060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758785+0000","last_change":"2023-07-08T12:38:08.557447+0000","last_active":"2023-07-08T12:38:13.758785+0000","last_peered":"2023-07-08T12:38:13.758785+0000","last_clean":"2023-07-08T12:38:13.758785+0000","last_became_active":"2023-07-08T12:38:08.557112+0000","last_became_peered":"2023-07-08T12:38:08.557112+0000","last_unstale":"2023-07-08T12:38:13.758785+0000","last_undegraded":"2023-07-08T12:38:13.758785+0000","last_fullsized":"2023-07-08T12:38:13.758785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:31:53.990746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606414+0000","last_change":"2023-07-08T12:38:08.550996+0000","last_active":"2023-07-08T12:38:13.606414+0000","last_peered":"2023-07-08T12:38:13.606414+0000","last_clean":"2023-07-08T12:38:13.606414+0000","last_became_active":"2023-07-08T12:38:08.550597+0000","last_became_peered":"2023-07-08T12:38:08.550597+0000","last_unstale":"2023-07-08T12:38:13.606414+0000","last_undegraded":"2023-07-08T12:38:13.606414+0000","last_fullsized":"2023-07-08T12:38:13.606414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:44:57.917258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758464+0000","last_change":"2023-07-08T12:38:08.552118+0000","last_active":"2023-07-08T12:38:13.758464+0000","last_peered":"2023-07-08T12:38:13.758464+0000","last_clean":"2023-07-08T12:38:13.758464+0000","last_became_active":"2023-07-08T12:38:08.551258+0000","last_became_peered":"2023-07-08T12:38:08.551258+0000","last_unstale":"2023-07-08T12:38:13.758464+0000","last_undegraded":"2023-07-08T12:38:13.758464+0000","last_fullsized":"2023-07-08T12:38:13.758464+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:22.667819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759628+0000","last_change":"2023-07-08T12:38:08.563050+0000","last_active":"2023-07-08T12:38:13.759628+0000","last_peered":"2023-07-08T12:38:13.759628+0000","last_clean":"2023-07-08T12:38:13.759628+0000","last_became_active":"2023-07-08T12:38:08.562779+0000","last_became_peered":"2023-07-08T12:38:08.562779+0000","last_unstale":"2023-07-08T12:38:13.759628+0000","last_undegraded":"2023-07-08T12:38:13.759628+0000","last_fullsized":"2023-07-08T12:38:13.759628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:15.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4592,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":718950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":901120,"data_stored":715522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":323584,"data_stored":128422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":311296,"data_stored":125224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-08T12:38:33.877 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-08T12:38:33.878 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-08T12:38:33.878 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-07-08T12:38:33.878 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-07-08T12:38:33.879 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-07-08T12:38:34.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.032+0000 7fdb547e7700 1 Processor -- start 2023-07-08T12:38:34.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.032+0000 7fdb547e7700 1 -- start start 2023-07-08T12:38:34.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb547e7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdb4c15a480 0x7fdb4c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb547e7700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdb4c0cfea0 0x7fdb4c0b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb547e7700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4c15a850 con 0x7fdb4c0b2410 2023-07-08T12:38:34.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb547e7700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4c0b2da0 con 0x7fdb4c15a480 2023-07-08T12:38:34.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb547e7700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4c0cde90 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb52583700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdb4c15a480 0x7fdb4c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb52583700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdb4c15a480 0x7fdb4c0cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49584/0 (socket says 172.21.15.12:49584) 2023-07-08T12:38:34.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb52583700 1 -- 172.21.15.12:0/1199293433 learned_addr learned my addr 172.21.15.12:0/1199293433 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:34.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb52d84700 1 --2- 172.21.15.12:0/1199293433 >> v2:172.21.15.160:3300/0 conn(0x7fdb4c0cfea0 0x7fdb4c0b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb52583700 1 -- 172.21.15.12:0/1199293433 >> v2:172.21.15.160:3300/0 conn(0x7fdb4c0cfea0 msgr2=0x7fdb4c0b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:34.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb52583700 1 --2- 172.21.15.12:0/1199293433 >> v2:172.21.15.160:3300/0 conn(0x7fdb4c0cfea0 0x7fdb4c0b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb52583700 1 -- 172.21.15.12:0/1199293433 >> v1:172.21.15.160:6789/0 conn(0x7fdb4c0b2410 legacy=0x7fdb4c15def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:34.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb52583700 1 -- 172.21.15.12:0/1199293433 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c0cd8d0 con 0x7fdb4c15a480 2023-07-08T12:38:34.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb52583700 1 --2- 172.21.15.12:0/1199293433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb4c15a480 0x7fdb4c0cafe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0cc720 tx=0x7fdb48009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6b41dda3b76d11 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:34.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb51581700 1 -- 172.21.15.12:0/1199293433 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48004230 con 0x7fdb4c15a480 2023-07-08T12:38:34.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb51581700 1 -- 172.21.15.12:0/1199293433 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb48004390 con 0x7fdb4c15a480 2023-07-08T12:38:34.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.033+0000 7fdb51581700 1 -- 172.21.15.12:0/1199293433 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480036a0 con 0x7fdb4c15a480 2023-07-08T12:38:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.034+0000 7fdb547e7700 1 -- 172.21.15.12:0/1199293433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb4c15a480 msgr2=0x7fdb4c0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.034+0000 7fdb547e7700 1 --2- 172.21.15.12:0/1199293433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb4c15a480 0x7fdb4c0cafe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0cc720 tx=0x7fdb48009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.034+0000 7fdb547e7700 1 -- 172.21.15.12:0/1199293433 shutdown_connections 2023-07-08T12:38:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.034+0000 7fdb547e7700 1 --2- 172.21.15.12:0/1199293433 >> v2:172.21.15.160:3300/0 conn(0x7fdb4c0cfea0 0x7fdb4c0b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.034+0000 7fdb547e7700 1 --2- 172.21.15.12:0/1199293433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb4c15a480 0x7fdb4c0cafe0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.034+0000 7fdb547e7700 1 -- 172.21.15.12:0/1199293433 >> 172.21.15.12:0/1199293433 conn(0x7fdb4c06ab60 msgr2=0x7fdb4c06e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.034+0000 7fdb547e7700 1 -- 172.21.15.12:0/1199293433 shutdown_connections 2023-07-08T12:38:34.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.034+0000 7fdb547e7700 1 -- 172.21.15.12:0/1199293433 wait complete. 2023-07-08T12:38:34.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.035+0000 7fdb547e7700 1 Processor -- start 2023-07-08T12:38:34.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.035+0000 7fdb547e7700 1 -- start start 2023-07-08T12:38:34.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb547e7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb4c0b2410 0x7fdb4c0a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb547e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb4c0cfea0 0x7fdb4c1e5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb547e7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb4c15a480 0x7fdb4c14dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb547e7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c1e5ab0 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb547e7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c1e5ce0 con 0x7fdb4c0b2410 2023-07-08T12:38:34.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb547e7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c1f1cf0 con 0x7fdb4c15a480 2023-07-08T12:38:34.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb51d82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb4c0cfea0 0x7fdb4c1e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb51d82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb4c0cfea0 0x7fdb4c1e5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49592/0 (socket says 172.21.15.12:49592) 2023-07-08T12:38:34.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb51d82700 1 -- 172.21.15.12:0/1152541488 learned_addr learned my addr 172.21.15.12:0/1152541488 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb52583700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb4c0b2410 0x7fdb4c0a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb52d84700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb4c15a480 0x7fdb4c14dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb51d82700 1 -- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb4c15a480 msgr2=0x7fdb4c14dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb51d82700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb4c15a480 0x7fdb4c14dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb51d82700 1 -- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb4c0b2410 msgr2=0x7fdb4c0a7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.036+0000 7fdb51d82700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb4c0b2410 0x7fdb4c0a7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.037+0000 7fdb51d82700 1 -- 172.21.15.12:0/1152541488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb48009550 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.037+0000 7fdb52d84700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb4c15a480 0x7fdb4c14dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.037+0000 7fdb51d82700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb4c0cfea0 0x7fdb4c1e5570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00b4a0 tx=0x7fdb3c00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.037+0000 7fdb42ffd700 1 -- 172.21.15.12:0/1152541488 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c011830 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.037+0000 7fdb42ffd700 1 -- 172.21.15.12:0/1152541488 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c011990 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.037+0000 7fdb547e7700 1 -- 172.21.15.12:0/1152541488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c06b170 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.037+0000 7fdb42ffd700 1 -- 172.21.15.12:0/1152541488 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c011d40 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.037+0000 7fdb547e7700 1 -- 172.21.15.12:0/1152541488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c06b730 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.038+0000 7fdb547e7700 1 -- 172.21.15.12:0/1152541488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c06d0b0 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.038+0000 7fdb42ffd700 1 -- 172.21.15.12:0/1152541488 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdb3c01a400 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.039+0000 7fdb42ffd700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdb3006ace0 0x7fdb3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.039+0000 7fdb42ffd700 1 -- 172.21.15.12:0/1152541488 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdb3c089430 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.039+0000 7fdb52583700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdb3006ace0 0x7fdb3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.039+0000 7fdb52583700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdb3006ace0 0x7fdb3006d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009130 tx=0x7fdb48008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:34.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.043+0000 7fdb42ffd700 1 -- 172.21.15.12:0/1152541488 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdb4c06d0b0 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.297+0000 7fdb547e7700 1 -- 172.21.15.12:0/1152541488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fdb4c06d0b0 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.300+0000 7fdb42ffd700 1 -- 172.21.15.12:0/1152541488 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170845 (secure 0 0 0) 0x7fdb4c06d0b0 con 0x7fdb4c0cfea0 2023-07-08T12:38:34.303 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:34.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.303+0000 7fdb547e7700 1 -- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdb3006ace0 msgr2=0x7fdb3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.303+0000 7fdb547e7700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdb3006ace0 0x7fdb3006d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009130 tx=0x7fdb48008030 comp rx=0 tx=0).stop 2023-07-08T12:38:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.303+0000 7fdb547e7700 1 -- 172.21.15.12:0/1152541488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb4c0cfea0 msgr2=0x7fdb4c1e5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.303+0000 7fdb547e7700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb4c0cfea0 0x7fdb4c1e5570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00b4a0 tx=0x7fdb3c00b7b0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.303+0000 7fdb547e7700 1 -- 172.21.15.12:0/1152541488 shutdown_connections 2023-07-08T12:38:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.303+0000 7fdb547e7700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdb3006ace0 0x7fdb3006d190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.303+0000 7fdb547e7700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb4c15a480 0x7fdb4c14dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.303+0000 7fdb547e7700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb4c0b2410 0x7fdb4c0a7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.303+0000 7fdb547e7700 1 --2- 172.21.15.12:0/1152541488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb4c0cfea0 0x7fdb4c1e5570 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.303+0000 7fdb547e7700 1 -- 172.21.15.12:0/1152541488 >> 172.21.15.12:0/1152541488 conn(0x7fdb4c06ab60 msgr2=0x7fdb4c0d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.303+0000 7fdb547e7700 1 -- 172.21.15.12:0/1152541488 shutdown_connections 2023-07-08T12:38:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.303+0000 7fdb547e7700 1 -- 172.21.15.12:0/1152541488 wait complete. 2023-07-08T12:38:34.316 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":5,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":550875184},{"type":"v1","addr":"172.21.15.160:6833","nonce":550875184}]},"active_addr":"172.21.15.160:6833/550875184","active_change":"2023-07-08T12:37:48.128212+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4101,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":610039537}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":2923652514}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":1518359367}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":3395862364}]}]} 2023-07-08T12:38:34.320 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-07-08T12:38:34.321 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-07-08T12:38:34.321 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:34.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.444+0000 7fabd9cb1700 1 Processor -- start 2023-07-08T12:38:34.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.445+0000 7fabd9cb1700 1 -- start start 2023-07-08T12:38:34.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.445+0000 7fabd9cb1700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd40b1c70 0x7fabd40ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.445+0000 7fabd9cb1700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fabd415d6c0 0x7fabd4071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.445+0000 7fabd9cb1700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd415d230 con 0x7fabd40ce1b0 2023-07-08T12:38:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.445+0000 7fabd9cb1700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd40c9e60 con 0x7fabd415d6c0 2023-07-08T12:38:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.446+0000 7fabd9cb1700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd40ceac0 con 0x7fabd40b1c70 2023-07-08T12:38:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.446+0000 7fabd3fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fabd415d6c0 0x7fabd4071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.446+0000 7fabd3fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fabd415d6c0 0x7fabd4071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49602/0 (socket says 172.21.15.12:49602) 2023-07-08T12:38:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.446+0000 7fabd3fff700 1 -- 172.21.15.12:0/3557854864 learned_addr learned my addr 172.21.15.12:0/3557854864 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.446+0000 7fabd2ffd700 1 --2- 172.21.15.12:0/3557854864 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd40b1c70 0x7fabd40ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.446+0000 7fabd3fff700 1 -- 172.21.15.12:0/3557854864 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd40b1c70 msgr2=0x7fabd40ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:34.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.446+0000 7fabd3fff700 1 --2- 172.21.15.12:0/3557854864 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd40b1c70 0x7fabd40ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.446+0000 7fabd3fff700 1 -- 172.21.15.12:0/3557854864 >> v1:172.21.15.160:6789/0 conn(0x7fabd40ce1b0 legacy=0x7fabd40cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.446+0000 7fabd3fff700 1 -- 172.21.15.12:0/3557854864 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd40cdbc0 con 0x7fabd415d6c0 2023-07-08T12:38:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.446+0000 7fabd3fff700 1 --2- 172.21.15.12:0/3557854864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabd415d6c0 0x7fabd4071530 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00d150 tx=0x7fabcc00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55c1dd4cfe809ea server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.446+0000 7fabd27fc700 1 -- 172.21.15.12:0/3557854864 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc010070 con 0x7fabd415d6c0 2023-07-08T12:38:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.446+0000 7fabd27fc700 1 -- 172.21.15.12:0/3557854864 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc00ddc0 con 0x7fabd415d6c0 2023-07-08T12:38:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.446+0000 7fabd27fc700 1 -- 172.21.15.12:0/3557854864 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc00b310 con 0x7fabd415d6c0 2023-07-08T12:38:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.447+0000 7fabd9cb1700 1 -- 172.21.15.12:0/3557854864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabd415d6c0 msgr2=0x7fabd4071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.447+0000 7fabd9cb1700 1 --2- 172.21.15.12:0/3557854864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabd415d6c0 0x7fabd4071530 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00d150 tx=0x7fabcc00d460 comp rx=0 tx=0).stop 2023-07-08T12:38:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.447+0000 7fabd9cb1700 1 -- 172.21.15.12:0/3557854864 shutdown_connections 2023-07-08T12:38:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.447+0000 7fabd9cb1700 1 --2- 172.21.15.12:0/3557854864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabd415d6c0 0x7fabd4071530 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.447+0000 7fabd9cb1700 1 --2- 172.21.15.12:0/3557854864 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd40b1c70 0x7fabd40ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.447+0000 7fabd9cb1700 1 -- 172.21.15.12:0/3557854864 >> 172.21.15.12:0/3557854864 conn(0x7fabd40cc290 msgr2=0x7fabd40cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.447+0000 7fabd9cb1700 1 -- 172.21.15.12:0/3557854864 shutdown_connections 2023-07-08T12:38:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.447+0000 7fabd9cb1700 1 -- 172.21.15.12:0/3557854864 wait complete. 2023-07-08T12:38:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.448+0000 7fabd9cb1700 1 Processor -- start 2023-07-08T12:38:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.448+0000 7fabd9cb1700 1 -- start start 2023-07-08T12:38:34.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.449+0000 7fabd9cb1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabd40b1c70 0x7fabd40c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.449+0000 7fabd9cb1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabd40ce1b0 0x7fabd40c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.449+0000 7fabd9cb1700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd415d6c0 0x7fabd41eec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.449+0000 7fabd9cb1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd40c81d0 con 0x7fabd40ce1b0 2023-07-08T12:38:34.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.449+0000 7fabd9cb1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd40c7130 con 0x7fabd40b1c70 2023-07-08T12:38:34.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.449+0000 7fabd9cb1700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd40c7410 con 0x7fabd415d6c0 2023-07-08T12:38:34.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.449+0000 7fabd2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabd40ce1b0 0x7fabd40c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.449+0000 7fabd2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabd40ce1b0 0x7fabd40c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49606/0 (socket says 172.21.15.12:49606) 2023-07-08T12:38:34.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.449+0000 7fabd2ffd700 1 -- 172.21.15.12:0/2748420243 learned_addr learned my addr 172.21.15.12:0/2748420243 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:34.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.449+0000 7fabd3fff700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd415d6c0 0x7fabd41eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.449+0000 7fabd37fe700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabd40b1c70 0x7fabd40c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.450+0000 7fabd2ffd700 1 -- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd415d6c0 msgr2=0x7fabd41eec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.450+0000 7fabd2ffd700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd415d6c0 0x7fabd41eec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.450+0000 7fabd2ffd700 1 -- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabd40b1c70 msgr2=0x7fabd40c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.450+0000 7fabd2ffd700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabd40b1c70 0x7fabd40c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.450+0000 7fabd2ffd700 1 -- 172.21.15.12:0/2748420243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc00bd40 con 0x7fabd40ce1b0 2023-07-08T12:38:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.450+0000 7fabd2ffd700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabd40ce1b0 0x7fabd40c8f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fabc800e810 tx=0x7fabc800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.450+0000 7fabbbfff700 1 -- 172.21.15.12:0/2748420243 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc800c070 con 0x7fabd40ce1b0 2023-07-08T12:38:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.450+0000 7fabbbfff700 1 -- 172.21.15.12:0/2748420243 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc8013610 con 0x7fabd40ce1b0 2023-07-08T12:38:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.450+0000 7fabd9cb1700 1 -- 172.21.15.12:0/2748420243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd40c76f0 con 0x7fabd40ce1b0 2023-07-08T12:38:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.450+0000 7fabbbfff700 1 -- 172.21.15.12:0/2748420243 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc80139c0 con 0x7fabd40ce1b0 2023-07-08T12:38:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.450+0000 7fabd9cb1700 1 -- 172.21.15.12:0/2748420243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd40c7cb0 con 0x7fabd40ce1b0 2023-07-08T12:38:34.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.451+0000 7fabd9cb1700 1 -- 172.21.15.12:0/2748420243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd40c6690 con 0x7fabd40ce1b0 2023-07-08T12:38:34.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.451+0000 7fabbbfff700 1 -- 172.21.15.12:0/2748420243 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fabc8013b20 con 0x7fabd40ce1b0 2023-07-08T12:38:34.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.452+0000 7fabbbfff700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabc006ace0 0x7fabc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.452+0000 7fabbbfff700 1 -- 172.21.15.12:0/2748420243 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fabc8089250 con 0x7fabd40ce1b0 2023-07-08T12:38:34.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.452+0000 7fabd37fe700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabc006ace0 0x7fabc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.453+0000 7fabd37fe700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabc006ace0 0x7fabc006d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fabc4007fd0 tx=0x7fabc4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:34.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.457+0000 7fabbbfff700 1 -- 172.21.15.12:0/2748420243 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fabc8057aa0 con 0x7fabd40ce1b0 2023-07-08T12:38:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.659+0000 7fabd9cb1700 1 -- 172.21.15.12:0/2748420243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabd404ea60 con 0x7fabd40ce1b0 2023-07-08T12:38:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.661+0000 7fabbbfff700 1 -- 172.21.15.12:0/2748420243 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20108 (secure 0 0 0) 0x7fabc8057630 con 0x7fabd40ce1b0 2023-07-08T12:38:34.663 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:34.664 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:12.993803+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.663+0000 7fabd9cb1700 1 -- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabc006ace0 msgr2=0x7fabc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.663+0000 7fabd9cb1700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabc006ace0 0x7fabc006d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fabc4007fd0 tx=0x7fabc4009040 comp rx=0 tx=0).stop 2023-07-08T12:38:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.663+0000 7fabd9cb1700 1 -- 172.21.15.12:0/2748420243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabd40ce1b0 msgr2=0x7fabd40c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.663+0000 7fabd9cb1700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabd40ce1b0 0x7fabd40c8f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fabc800e810 tx=0x7fabc800eb20 comp rx=0 tx=0).stop 2023-07-08T12:38:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.663+0000 7fabd9cb1700 1 -- 172.21.15.12:0/2748420243 shutdown_connections 2023-07-08T12:38:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.663+0000 7fabd9cb1700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabc006ace0 0x7fabc006d190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.663+0000 7fabd9cb1700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd415d6c0 0x7fabd41eec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.663+0000 7fabd9cb1700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabd40b1c70 0x7fabd40c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.663+0000 7fabd9cb1700 1 --2- 172.21.15.12:0/2748420243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabd40ce1b0 0x7fabd40c8f70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.663+0000 7fabd9cb1700 1 -- 172.21.15.12:0/2748420243 >> 172.21.15.12:0/2748420243 conn(0x7fabd40cc290 msgr2=0x7fabd40af270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.663+0000 7fabd9cb1700 1 -- 172.21.15.12:0/2748420243 shutdown_connections 2023-07-08T12:38:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.663+0000 7fabd9cb1700 1 -- 172.21.15.12:0/2748420243 wait complete. 2023-07-08T12:38:34.676 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-07-08T12:38:34.677 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:34.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.800+0000 7fe9910a9700 1 Processor -- start 2023-07-08T12:38:34.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.800+0000 7fe9910a9700 1 -- start start 2023-07-08T12:38:34.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.801+0000 7fe9910a9700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe98c070bb0 0x7fe98c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.801+0000 7fe9910a9700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c0cd530 0x7fe98c070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.801+0000 7fe9910a9700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98c06b1b0 con 0x7fe98c06b640 2023-07-08T12:38:34.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.801+0000 7fe9910a9700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98c15d1f0 con 0x7fe98c070bb0 2023-07-08T12:38:34.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.801+0000 7fe9910a9700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0714c0 con 0x7fe98c0cd530 2023-07-08T12:38:34.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.801+0000 7fe98b59e700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe98c06b640 0x7fe98c06cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37010/0 (socket says 172.21.15.12:37010) 2023-07-08T12:38:34.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.801+0000 7fe98b59e700 1 -- 172.21.15.12:0/251931276 learned_addr learned my addr 172.21.15.12:0/251931276 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:34.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.801+0000 7fe98a59c700 1 --2- 172.21.15.12:0/251931276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c0cd530 0x7fe98c070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.801+0000 7fe98ad9d700 1 --2- 172.21.15.12:0/251931276 >> v2:172.21.15.160:3300/0 conn(0x7fe98c070bb0 0x7fe98c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763971051 0 0) 0x7fe98c06b1b0 con 0x7fe98c06b640 2023-07-08T12:38:34.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9740035b0 con 0x7fe98c06b640 2023-07-08T12:38:34.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3493533363 0 0) 0x7fe9740035b0 con 0x7fe98c06b640 2023-07-08T12:38:34.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c0cd530 msgr2=0x7fe98c070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:34.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 --2- 172.21.15.12:0/251931276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c0cd530 0x7fe98c070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 >> v2:172.21.15.160:3300/0 conn(0x7fe98c070bb0 msgr2=0x7fe98c15ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:34.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 --2- 172.21.15.12:0/251931276 >> v2:172.21.15.160:3300/0 conn(0x7fe98c070bb0 0x7fe98c15ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c06a5a0 con 0x7fe98c06b640 2023-07-08T12:38:34.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7fe9840019f0 con 0x7fe98c06b640 2023-07-08T12:38:34.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 >> v1:172.21.15.12:6789/0 conn(0x7fe98c06b640 legacy=0x7fe98c06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 --2- 172.21.15.12:0/251931276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974003a00 0x7fe974003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 --2- 172.21.15.12:0/251931276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe974004430 0x7fe9740048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 --2- 172.21.15.12:0/251931276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c0cd530 0x7fe974004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c15d1f0 con 0x7fe974003a00 2023-07-08T12:38:34.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0714c0 con 0x7fe974004430 2023-07-08T12:38:34.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97400ba10 con 0x7fe98c0cd530 2023-07-08T12:38:34.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe98ad9d700 1 --2- 172.21.15.12:0/251931276 >> v2:172.21.15.160:3300/0 conn(0x7fe98c070bb0 0x7fe98c15ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:38:34.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe98a59c700 1 --2- 172.21.15.12:0/251931276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe974004430 0x7fe9740048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe98b59e700 1 --2- 172.21.15.12:0/251931276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c0cd530 0x7fe974004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe98ad9d700 1 --2- 172.21.15.12:0/251931276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974003a00 0x7fe974003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe98ad9d700 1 -- 172.21.15.12:0/251931276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c0cd530 msgr2=0x7fe974004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:34.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe98ad9d700 1 --2- 172.21.15.12:0/251931276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c0cd530 0x7fe974004ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe98ad9d700 1 -- 172.21.15.12:0/251931276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe974004430 msgr2=0x7fe9740048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:34.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe98ad9d700 1 --2- 172.21.15.12:0/251931276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe974004430 0x7fe9740048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.802+0000 7fe98ad9d700 1 -- 172.21.15.12:0/251931276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9740035b0 con 0x7fe974003a00 2023-07-08T12:38:34.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.803+0000 7fe98ad9d700 1 --2- 172.21.15.12:0/251931276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974003a00 0x7fe974003e10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe980009d10 tx=0x7fe974005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d37a73c2cab0f15 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:34.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.803+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980009e60 con 0x7fe974003a00 2023-07-08T12:38:34.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.803+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98000e9a0 con 0x7fe974003a00 2023-07-08T12:38:34.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.803+0000 7fe989d9b700 1 -- 172.21.15.12:0/251931276 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98000ed10 con 0x7fe974003a00 2023-07-08T12:38:34.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.803+0000 7fe9910a9700 1 -- 172.21.15.12:0/251931276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974003a00 msgr2=0x7fe974003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.803+0000 7fe9910a9700 1 --2- 172.21.15.12:0/251931276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974003a00 0x7fe974003e10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe980009d10 tx=0x7fe974005880 comp rx=0 tx=0).stop 2023-07-08T12:38:34.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.803+0000 7fe98b59e700 1 -- 172.21.15.12:0/251931276 reap_dead start 2023-07-08T12:38:34.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.803+0000 7fe9910a9700 1 -- 172.21.15.12:0/251931276 shutdown_connections 2023-07-08T12:38:34.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.803+0000 7fe9910a9700 1 -- 172.21.15.12:0/251931276 >> 172.21.15.12:0/251931276 conn(0x7fe98c0cb9f0 msgr2=0x7fe98c0cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:34.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.803+0000 7fe9910a9700 1 -- 172.21.15.12:0/251931276 shutdown_connections 2023-07-08T12:38:34.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.803+0000 7fe9910a9700 1 -- 172.21.15.12:0/251931276 wait complete. 2023-07-08T12:38:34.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.805+0000 7fe9910a9700 1 Processor -- start 2023-07-08T12:38:34.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.805+0000 7fe9910a9700 1 -- start start 2023-07-08T12:38:34.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe9910a9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974003a00 0x7fe98c1ed920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe9910a9700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c14b7f0 0x7fe98c14bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe9910a9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe98c0c7350 0x7fe98c0c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe9910a9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c14c4c0 con 0x7fe974003a00 2023-07-08T12:38:34.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe9910a9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c14c7a0 con 0x7fe98c0c7350 2023-07-08T12:38:34.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe9910a9700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c14ca80 con 0x7fe98c14b7f0 2023-07-08T12:38:34.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe98ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974003a00 0x7fe98c1ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe98ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974003a00 0x7fe98c1ed920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49624/0 (socket says 172.21.15.12:49624) 2023-07-08T12:38:34.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe98ad9d700 1 -- 172.21.15.12:0/418671774 learned_addr learned my addr 172.21.15.12:0/418671774 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:34.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe98a59c700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c14b7f0 0x7fe98c14bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe98b59e700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe98c0c7350 0x7fe98c0c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe98ad9d700 1 -- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c14b7f0 msgr2=0x7fe98c14bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe98ad9d700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c14b7f0 0x7fe98c14bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe98ad9d700 1 -- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe98c0c7350 msgr2=0x7fe98c0c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:34.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe98ad9d700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe98c0c7350 0x7fe98c0c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:34.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe98ad9d700 1 -- 172.21.15.12:0/418671774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c14cd60 con 0x7fe974003a00 2023-07-08T12:38:34.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe98a59c700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c14b7f0 0x7fe98c14bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:34.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe98ad9d700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974003a00 0x7fe98c1ed920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe974005790 tx=0x7fe9800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:34.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe9737fe700 1 -- 172.21.15.12:0/418671774 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9800038a0 con 0x7fe974003a00 2023-07-08T12:38:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe9737fe700 1 -- 172.21.15.12:0/418671774 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe980003a20 con 0x7fe974003a00 2023-07-08T12:38:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe9910a9700 1 -- 172.21.15.12:0/418671774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98c14c1a0 con 0x7fe974003a00 2023-07-08T12:38:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe9737fe700 1 -- 172.21.15.12:0/418671774 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980003dd0 con 0x7fe974003a00 2023-07-08T12:38:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.806+0000 7fe9910a9700 1 -- 172.21.15.12:0/418671774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98c149f20 con 0x7fe974003a00 2023-07-08T12:38:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.807+0000 7fe9910a9700 1 -- 172.21.15.12:0/418671774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98c0c71a0 con 0x7fe974003a00 2023-07-08T12:38:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.808+0000 7fe9737fe700 1 -- 172.21.15.12:0/418671774 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe980013470 con 0x7fe974003a00 2023-07-08T12:38:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.808+0000 7fe9737fe700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe97806ace0 0x7fe97806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.808+0000 7fe9737fe700 1 -- 172.21.15.12:0/418671774 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe98008cb90 con 0x7fe974003a00 2023-07-08T12:38:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.809+0000 7fe98a59c700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe97806ace0 0x7fe97806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:34.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.809+0000 7fe98a59c700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe97806ace0 0x7fe97806d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0c90b0 tx=0x7fe97c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:34.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:34.813+0000 7fe9737fe700 1 -- 172.21.15.12:0/418671774 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe9800579e0 con 0x7fe974003a00 2023-07-08T12:38:35.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.011+0000 7fe9910a9700 1 -- 172.21.15.12:0/418671774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe98c04ea60 con 0x7fe974003a00 2023-07-08T12:38:35.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.013+0000 7fe9737fe700 1 -- 172.21.15.12:0/418671774 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20108 (secure 0 0 0) 0x7fe98005b000 con 0x7fe974003a00 2023-07-08T12:38:35.014 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:35.015 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:12.993803+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:35.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.015+0000 7fe9910a9700 1 -- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe97806ace0 msgr2=0x7fe97806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.015+0000 7fe9910a9700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe97806ace0 0x7fe97806d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0c90b0 tx=0x7fe97c008040 comp rx=0 tx=0).stop 2023-07-08T12:38:35.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.015+0000 7fe9910a9700 1 -- 172.21.15.12:0/418671774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974003a00 msgr2=0x7fe98c1ed920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.015+0000 7fe9910a9700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974003a00 0x7fe98c1ed920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe974005790 tx=0x7fe9800047c0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.015+0000 7fe9910a9700 1 -- 172.21.15.12:0/418671774 shutdown_connections 2023-07-08T12:38:35.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.015+0000 7fe9910a9700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe97806ace0 0x7fe97806d190 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.015+0000 7fe9910a9700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe98c0c7350 0x7fe98c0c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.015+0000 7fe9910a9700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe974003a00 0x7fe98c1ed920 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.015+0000 7fe9910a9700 1 --2- 172.21.15.12:0/418671774 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe98c14b7f0 0x7fe98c14bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.015+0000 7fe9910a9700 1 -- 172.21.15.12:0/418671774 >> 172.21.15.12:0/418671774 conn(0x7fe98c0cb9f0 msgr2=0x7fe98c0cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.015+0000 7fe9910a9700 1 -- 172.21.15.12:0/418671774 shutdown_connections 2023-07-08T12:38:35.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.015+0000 7fe9910a9700 1 -- 172.21.15.12:0/418671774 wait complete. 2023-07-08T12:38:35.029 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-07-08T12:38:35.030 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-07-08T12:38:35.031 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-07-08T12:38:35.032 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-07-08T12:38:35.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-07-08T12:38:35.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-07-08T12:38:35.034 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-07-08T12:38:35.034 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-07-08T12:38:35.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.243+0000 7faf0ceac700 1 Processor -- start 2023-07-08T12:38:35.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.243+0000 7faf0ceac700 1 -- start start 2023-07-08T12:38:35.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.244+0000 7faf0ceac700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf080cda10 0x7faf0816c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.244+0000 7faf0ceac700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf080cb8a0 0x7faf0806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.244+0000 7faf0ceac700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf080ccc50 con 0x7faf080ce0e0 2023-07-08T12:38:35.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.244+0000 7faf0ceac700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf08089e70 con 0x7faf080cda10 2023-07-08T12:38:35.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.244+0000 7faf0ceac700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf080caac0 con 0x7faf080cb8a0 2023-07-08T12:38:35.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.245+0000 7fce4a80b700 1 Processor -- start 2023-07-08T12:38:35.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.245+0000 7fce4a80b700 1 -- start start 2023-07-08T12:38:35.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.245+0000 7faf06d9d700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf080cb8a0 0x7faf0806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.245+0000 7faf06d9d700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf080cb8a0 0x7faf0806c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48378/0 (socket says 172.21.15.12:48378) 2023-07-08T12:38:35.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.245+0000 7faf06d9d700 1 -- 172.21.15.12:0/4081136401 learned_addr learned my addr 172.21.15.12:0/4081136401 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.245+0000 7faf06d9d700 1 -- 172.21.15.12:0/4081136401 >> v2:172.21.15.12:3300/0 conn(0x7faf080cda10 msgr2=0x7faf0816c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.245+0000 7faf06d9d700 1 --2- 172.21.15.12:0/4081136401 >> v2:172.21.15.12:3300/0 conn(0x7faf080cda10 0x7faf0816c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.245+0000 7faf06d9d700 1 -- 172.21.15.12:0/4081136401 >> v1:172.21.15.160:6789/0 conn(0x7faf080ce0e0 legacy=0x7faf080cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.245+0000 7faf06d9d700 1 -- 172.21.15.12:0/4081136401 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf080ccf30 con 0x7faf080cb8a0 2023-07-08T12:38:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.246+0000 7fce4a80b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fce4406c550 0x7fce440c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.246+0000 7fce4a80b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce440cb730 0x7fce4406c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.246+0000 7fce4a80b700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4415a5c0 con 0x7fce4415e490 2023-07-08T12:38:35.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.246+0000 7fce4a80b700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4415dd00 con 0x7fce4406c550 2023-07-08T12:38:35.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.246+0000 7fce4a80b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4406ce60 con 0x7fce440cb730 2023-07-08T12:38:35.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.245+0000 7faf06d9d700 1 --2- 172.21.15.12:0/4081136401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf080cb8a0 0x7faf0806c3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faf080cc320 tx=0x7faef8009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=265b5c9bd6cecdb server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.246+0000 7faf0559a700 1 -- 172.21.15.12:0/4081136401 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8004230 con 0x7faf080cb8a0 2023-07-08T12:38:35.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.246+0000 7fce437fe700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce440cb730 0x7fce4406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.246+0000 7faf0559a700 1 -- 172.21.15.12:0/4081136401 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef8004390 con 0x7faf080cb8a0 2023-07-08T12:38:35.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.246+0000 7faf0559a700 1 -- 172.21.15.12:0/4081136401 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef80036a0 con 0x7faf080cb8a0 2023-07-08T12:38:35.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.247+0000 7faf0ceac700 1 -- 172.21.15.12:0/4081136401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf080cb8a0 msgr2=0x7faf0806c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.247+0000 7faf0ceac700 1 --2- 172.21.15.12:0/4081136401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf080cb8a0 0x7faf0806c3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faf080cc320 tx=0x7faef8009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.247+0000 7fce437fe700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce440cb730 0x7fce4406c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59256/0 (socket says 172.21.15.12:59256) 2023-07-08T12:38:35.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.247+0000 7fce437fe700 1 -- 172.21.15.12:0/2400621125 learned_addr learned my addr 172.21.15.12:0/2400621125 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.248+0000 7faf0ceac700 1 -- 172.21.15.12:0/4081136401 shutdown_connections 2023-07-08T12:38:35.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.248+0000 7faf0ceac700 1 --2- 172.21.15.12:0/4081136401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf080cb8a0 0x7faf0806c3f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.248+0000 7faf0ceac700 1 --2- 172.21.15.12:0/4081136401 >> v2:172.21.15.12:3300/0 conn(0x7faf080cda10 0x7faf0816c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.248+0000 7faf0ceac700 1 -- 172.21.15.12:0/4081136401 >> 172.21.15.12:0/4081136401 conn(0x7faf0806abc0 msgr2=0x7faf080b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.248+0000 7faf0ceac700 1 -- 172.21.15.12:0/4081136401 shutdown_connections 2023-07-08T12:38:35.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.248+0000 7faf0ceac700 1 -- 172.21.15.12:0/4081136401 wait complete. 2023-07-08T12:38:35.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.250+0000 7faf0ceac700 1 Processor -- start 2023-07-08T12:38:35.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.250+0000 7faf0ceac700 1 -- start start 2023-07-08T12:38:35.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.250+0000 7fce437fe700 1 -- 172.21.15.12:0/2400621125 >> v2:172.21.15.12:3300/0 conn(0x7fce4406c550 msgr2=0x7fce440c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.250+0000 7fce437fe700 1 --2- 172.21.15.12:0/2400621125 >> v2:172.21.15.12:3300/0 conn(0x7fce4406c550 0x7fce440c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.250+0000 7fce437fe700 1 -- 172.21.15.12:0/2400621125 >> v1:172.21.15.12:6789/0 conn(0x7fce4415e490 legacy=0x7fce440c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.250+0000 7fce437fe700 1 -- 172.21.15.12:0/2400621125 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4406e990 con 0x7fce440cb730 2023-07-08T12:38:35.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.251+0000 7fce437fe700 1 --2- 172.21.15.12:0/2400621125 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce440cb730 0x7fce4406c920 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fce380098f0 tx=0x7fce38009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96574ab020760ca1 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.251+0000 7faf0ceac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf080cb8a0 0x7faf08089550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.251+0000 7faf0ceac700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf080cda10 0x7faf080b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.251+0000 7faf0ceac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf080ce0e0 0x7faf0816dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.251+0000 7faf0ceac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf080b2af0 con 0x7faf080ce0e0 2023-07-08T12:38:35.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.251+0000 7faf0ceac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf080b2dd0 con 0x7faf080cb8a0 2023-07-08T12:38:35.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.251+0000 7faf0ceac700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf080b30b0 con 0x7faf080cda10 2023-07-08T12:38:35.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.251+0000 7faf05d9b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf080cda10 0x7faf080b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.251+0000 7faf05d9b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf080cda10 0x7faf080b20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59266/0 (socket says 172.21.15.12:59266) 2023-07-08T12:38:35.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.251+0000 7faf05d9b700 1 -- 172.21.15.12:0/2479572126 learned_addr learned my addr 172.21.15.12:0/2479572126 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.252+0000 7faf06d9d700 1 --2- 172.21.15.12:0/2479572126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf080ce0e0 0x7faf0816dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.252+0000 7faf05d9b700 1 -- 172.21.15.12:0/2479572126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf080cb8a0 msgr2=0x7faf08089550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.252+0000 7faf05d9b700 1 --2- 172.21.15.12:0/2479572126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf080cb8a0 0x7faf08089550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.252+0000 7faf05d9b700 1 -- 172.21.15.12:0/2479572126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf080ce0e0 msgr2=0x7faf0816dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.252+0000 7faf05d9b700 1 --2- 172.21.15.12:0/2479572126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf080ce0e0 0x7faf0816dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.252+0000 7faf05d9b700 1 -- 172.21.15.12:0/2479572126 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef8009550 con 0x7faf080cda10 2023-07-08T12:38:35.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.252+0000 7faf05d9b700 1 --2- 172.21.15.12:0/2479572126 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf080cda10 0x7faf080b20a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf0000ba50 tx=0x7faf0000bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.252+0000 7faef6ffd700 1 -- 172.21.15.12:0/2479572126 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0000c9b0 con 0x7faf080cda10 2023-07-08T12:38:35.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.252+0000 7faef6ffd700 1 -- 172.21.15.12:0/2479572126 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0000cb50 con 0x7faf080cda10 2023-07-08T12:38:35.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.252+0000 7faf0ceac700 1 -- 172.21.15.12:0/2479572126 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf080c2030 con 0x7faf080cda10 2023-07-08T12:38:35.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.252+0000 7faf0ceac700 1 -- 172.21.15.12:0/2479572126 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf080c72b0 con 0x7faf080cda10 2023-07-08T12:38:35.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.253+0000 7faef6ffd700 1 -- 172.21.15.12:0/2479572126 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00014440 con 0x7faf080cda10 2023-07-08T12:38:35.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.253+0000 7faf0ceac700 1 -- 172.21.15.12:0/2479572126 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf080cd4f0 con 0x7faf080cda10 2023-07-08T12:38:35.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.254+0000 7fce42ffd700 1 -- 172.21.15.12:0/2400621125 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38004230 con 0x7fce440cb730 2023-07-08T12:38:35.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.254+0000 7fce42ffd700 1 -- 172.21.15.12:0/2400621125 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce38004390 con 0x7fce440cb730 2023-07-08T12:38:35.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.254+0000 7fce42ffd700 1 -- 172.21.15.12:0/2400621125 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce380036a0 con 0x7fce440cb730 2023-07-08T12:38:35.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.255+0000 7fce4a80b700 1 -- 172.21.15.12:0/2400621125 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce440cb730 msgr2=0x7fce4406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.255+0000 7fce4a80b700 1 --2- 172.21.15.12:0/2400621125 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce440cb730 0x7fce4406c920 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fce380098f0 tx=0x7fce38009c00 comp rx=0 tx=0).stop 2023-07-08T12:38:35.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.255+0000 7fce4a80b700 1 -- 172.21.15.12:0/2400621125 shutdown_connections 2023-07-08T12:38:35.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.255+0000 7fce4a80b700 1 --2- 172.21.15.12:0/2400621125 >> v2:172.21.15.12:3300/0 conn(0x7fce4406c550 0x7fce440c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.255+0000 7fce4a80b700 1 --2- 172.21.15.12:0/2400621125 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce440cb730 0x7fce4406c920 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.255+0000 7fce4a80b700 1 -- 172.21.15.12:0/2400621125 >> 172.21.15.12:0/2400621125 conn(0x7fce4406abc0 msgr2=0x7fce440b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.255+0000 7fce4a80b700 1 -- 172.21.15.12:0/2400621125 shutdown_connections 2023-07-08T12:38:35.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.255+0000 7fce4a80b700 1 -- 172.21.15.12:0/2400621125 wait complete. 2023-07-08T12:38:35.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.255+0000 7faef6ffd700 1 -- 172.21.15.12:0/2479572126 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faf000145a0 con 0x7faf080cda10 2023-07-08T12:38:35.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.256+0000 7faef6ffd700 1 --2- 172.21.15.12:0/2479572126 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faee806ace0 0x7faee806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.256+0000 7faef6ffd700 1 -- 172.21.15.12:0/2479572126 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faf00089ed0 con 0x7faf080cda10 2023-07-08T12:38:35.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.256+0000 7faef6ffd700 1 --2- 172.21.15.12:0/2479572126 >> [v2:172.21.15.12:6800/911521993,v1:172.21.15.12:6801/911521993] conn(0x7faee8074930 0x7faee8076d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.256+0000 7faef6ffd700 1 -- 172.21.15.12:0/2479572126 --> [v2:172.21.15.12:6800/911521993,v1:172.21.15.12:6801/911521993] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faee8077430 con 0x7faee8074930 2023-07-08T12:38:35.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.256+0000 7faef6ffd700 1 -- 172.21.15.12:0/2479572126 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf0008a320 con 0x7faf080cda10 2023-07-08T12:38:35.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.256+0000 7faf06d9d700 1 --2- 172.21.15.12:0/2479572126 >> [v2:172.21.15.12:6800/911521993,v1:172.21.15.12:6801/911521993] conn(0x7faee8074930 0x7faee8076d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.257+0000 7fce4a80b700 1 Processor -- start 2023-07-08T12:38:35.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.257+0000 7faf06d9d700 1 --2- 172.21.15.12:0/2479572126 >> [v2:172.21.15.12:6800/911521993,v1:172.21.15.12:6801/911521993] conn(0x7faee8074930 0x7faee8076d60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.258+0000 7faef6ffd700 1 -- 172.21.15.12:0/2479572126 <== osd.1 v2:172.21.15.12:6800/911521993 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7faee8077430 con 0x7faee8074930 2023-07-08T12:38:35.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.257+0000 7fce4a80b700 1 -- start start 2023-07-08T12:38:35.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.258+0000 7fce4a80b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce4406c550 0x7fce4415a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.258+0000 7fce4a80b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce440cb730 0x7fce4415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.258+0000 7fce4a80b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce4415e490 0x7fce4415d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.258+0000 7fce4a80b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4415b0d0 con 0x7fce4415e490 2023-07-08T12:38:35.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.258+0000 7fce4a80b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4415b3b0 con 0x7fce4406c550 2023-07-08T12:38:35.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.258+0000 7fce4a80b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4415b690 con 0x7fce440cb730 2023-07-08T12:38:35.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.259+0000 7fce43fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce4406c550 0x7fce4415a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.259+0000 7fce43fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce4406c550 0x7fce4415a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48400/0 (socket says 172.21.15.12:48400) 2023-07-08T12:38:35.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.259+0000 7fce43fff700 1 -- 172.21.15.12:0/2769713395 learned_addr learned my addr 172.21.15.12:0/2769713395 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.259+0000 7fce43fff700 1 -- 172.21.15.12:0/2769713395 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce440cb730 msgr2=0x7fce4415ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.259+0000 7fce43fff700 1 --2- 172.21.15.12:0/2769713395 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce440cb730 0x7fce4415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.259+0000 7fce43fff700 1 -- 172.21.15.12:0/2769713395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce4415e490 msgr2=0x7fce4415d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.259+0000 7fce43fff700 1 --2- 172.21.15.12:0/2769713395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce4415e490 0x7fce4415d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.259+0000 7fce43fff700 1 -- 172.21.15.12:0/2769713395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce38009550 con 0x7fce4406c550 2023-07-08T12:38:35.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.259+0000 7fce43fff700 1 --2- 172.21.15.12:0/2769713395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce4406c550 0x7fce4415a760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00ba50 tx=0x7fce3c00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.260+0000 7faf0659c700 1 --2- 172.21.15.12:0/2479572126 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faee806ace0 0x7faee806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.260+0000 7f9707b7f700 1 Processor -- start 2023-07-08T12:38:35.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.260+0000 7fb60ac49700 1 Processor -- start 2023-07-08T12:38:35.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.260+0000 7fb60ac49700 1 -- start start 2023-07-08T12:38:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.260+0000 7faf0659c700 1 --2- 172.21.15.12:0/2479572126 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faee806ace0 0x7faee806d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faefc0097b0 tx=0x7faefc006d20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.261+0000 7fb60ac49700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb60415f7f0 0x7fb6040cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.261+0000 7fb60ac49700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb6040b2b20 0x7fb6040ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.261+0000 7fb60ac49700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6040cd080 con 0x7fb6040ccbe0 2023-07-08T12:38:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.261+0000 7fb60ac49700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6040b3300 con 0x7fb60415f7f0 2023-07-08T12:38:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.261+0000 7fb60ac49700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6040cd360 con 0x7fb6040b2b20 2023-07-08T12:38:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.260+0000 7f9707b7f700 1 -- start start 2023-07-08T12:38:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f9707b7f700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f97000b1c70 0x7f97000ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f9707b7f700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f970015d6c0 0x7f9700071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f9707b7f700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f970015d230 con 0x7f97000ce1b0 2023-07-08T12:38:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f9707b7f700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97000c9e60 con 0x7f97000b1c70 2023-07-08T12:38:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f9707b7f700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97000ceac0 con 0x7f970015d6c0 2023-07-08T12:38:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f970591b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f97000ce1b0 0x7f97000cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37058/0 (socket says 172.21.15.12:37058) 2023-07-08T12:38:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f970591b700 1 -- 172.21.15.12:0/4167204430 learned_addr learned my addr 172.21.15.12:0/4167204430 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7fb6091e6700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb6040b2b20 0x7fb6040ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7fb6091e6700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb6040b2b20 0x7fb6040ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48404/0 (socket says 172.21.15.12:48404) 2023-07-08T12:38:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7fb6091e6700 1 -- 172.21.15.12:0/3230281903 learned_addr learned my addr 172.21.15.12:0/3230281903 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f970511a700 1 --2- 172.21.15.12:0/4167204430 >> v2:172.21.15.12:3300/0 conn(0x7f97000b1c70 0x7f97000ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f970611c700 1 --2- 172.21.15.12:0/4167204430 >> v2:172.21.15.160:3300/0 conn(0x7f970015d6c0 0x7f9700071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f970511a700 1 -- 172.21.15.12:0/4167204430 >> v2:172.21.15.160:3300/0 conn(0x7f970015d6c0 msgr2=0x7f9700071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f970511a700 1 --2- 172.21.15.12:0/4167204430 >> v2:172.21.15.160:3300/0 conn(0x7f970015d6c0 0x7f9700071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f970511a700 1 -- 172.21.15.12:0/4167204430 >> v1:172.21.15.12:6789/0 conn(0x7f97000ce1b0 legacy=0x7f97000cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f970511a700 1 -- 172.21.15.12:0/4167204430 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97000cdbc0 con 0x7f97000b1c70 2023-07-08T12:38:35.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7fb6091e6700 1 -- 172.21.15.12:0/3230281903 >> v2:172.21.15.12:3300/0 conn(0x7fb60415f7f0 msgr2=0x7fb6040cafe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-08T12:38:35.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7fb6091e6700 1 --2- 172.21.15.12:0/3230281903 >> v2:172.21.15.12:3300/0 conn(0x7fb60415f7f0 0x7fb6040cafe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7fb6091e6700 1 -- 172.21.15.12:0/3230281903 >> v1:172.21.15.160:6789/0 conn(0x7fb6040ccbe0 legacy=0x7fb6040b2410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-08T12:38:35.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7fb6091e6700 1 -- 172.21.15.12:0/3230281903 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6040cd640 con 0x7fb6040b2b20 2023-07-08T12:38:35.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7fb6091e6700 1 --2- 172.21.15.12:0/3230281903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6040b2b20 0x7fb6040ce660 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6040cc160 tx=0x7fb5f4009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7dede10b211adb6 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.263+0000 7fb6037fe700 1 -- 172.21.15.12:0/3230281903 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4004230 con 0x7fb6040b2b20 2023-07-08T12:38:35.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.263+0000 7fb6037fe700 1 -- 172.21.15.12:0/3230281903 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f4004390 con 0x7fb6040b2b20 2023-07-08T12:38:35.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.263+0000 7fb6037fe700 1 -- 172.21.15.12:0/3230281903 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40036a0 con 0x7fb6040b2b20 2023-07-08T12:38:35.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7fb60ac49700 1 -- 172.21.15.12:0/3230281903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6040b2b20 msgr2=0x7fb6040ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7fb60ac49700 1 --2- 172.21.15.12:0/3230281903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6040b2b20 0x7fb6040ce660 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6040cc160 tx=0x7fb5f4009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7fb60ac49700 1 -- 172.21.15.12:0/3230281903 shutdown_connections 2023-07-08T12:38:35.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7fb60ac49700 1 --2- 172.21.15.12:0/3230281903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6040b2b20 0x7fb6040ce660 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7fb60ac49700 1 --2- 172.21.15.12:0/3230281903 >> v2:172.21.15.12:3300/0 conn(0x7fb60415f7f0 0x7fb6040cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f4216ee9700 1 Processor -- start 2023-07-08T12:38:35.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.262+0000 7f4216ee9700 1 -- start start 2023-07-08T12:38:35.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7f4216ee9700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f42100c9970 0x7f42100cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7f4216ee9700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f42100c9d40 0x7f42100cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7f4216ee9700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42100cce20 con 0x7f42100cb6e0 2023-07-08T12:38:35.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7f4216ee9700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42100cd100 con 0x7f42100c9d40 2023-07-08T12:38:35.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7f4216ee9700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42100cd3e0 con 0x7f42100c9970 2023-07-08T12:38:35.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7fb60ac49700 1 -- 172.21.15.12:0/3230281903 >> 172.21.15.12:0/3230281903 conn(0x7fb60406ab60 msgr2=0x7fb60406e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7f4215486700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f42100cb6e0 0x7f4210157d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:46288/0 (socket says 172.21.15.12:46288) 2023-07-08T12:38:35.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7f4215486700 1 -- 172.21.15.12:0/3000091410 learned_addr learned my addr 172.21.15.12:0/3000091410 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:35.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7fce40ff9700 1 -- 172.21.15.12:0/2769713395 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c00cab0 con 0x7fce4406c550 2023-07-08T12:38:35.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7fce40ff9700 1 -- 172.21.15.12:0/2769713395 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c00cc30 con 0x7fce4406c550 2023-07-08T12:38:35.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7fce40ff9700 1 -- 172.21.15.12:0/2769713395 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c014440 con 0x7fce4406c550 2023-07-08T12:38:35.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7f4214c85700 1 --2- 172.21.15.12:0/3000091410 >> v2:172.21.15.160:3300/0 conn(0x7f42100c9970 0x7f42100cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7fce4a80b700 1 -- 172.21.15.12:0/2769713395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4415b970 con 0x7fce4406c550 2023-07-08T12:38:35.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.264+0000 7fce4a80b700 1 -- 172.21.15.12:0/2769713395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4415c170 con 0x7fce4406c550 2023-07-08T12:38:35.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.265+0000 7f420ffff700 1 --2- 172.21.15.12:0/3000091410 >> v2:172.21.15.12:3300/0 conn(0x7f42100c9d40 0x7f42100cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.265+0000 7f4214c85700 1 -- 172.21.15.12:0/3000091410 >> v2:172.21.15.12:3300/0 conn(0x7f42100c9d40 msgr2=0x7f42100cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.265+0000 7f4214c85700 1 --2- 172.21.15.12:0/3000091410 >> v2:172.21.15.12:3300/0 conn(0x7f42100c9d40 0x7f42100cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.265+0000 7f4214c85700 1 -- 172.21.15.12:0/3000091410 >> v1:172.21.15.160:6789/0 conn(0x7f42100cb6e0 legacy=0x7f4210157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.265+0000 7f4214c85700 1 -- 172.21.15.12:0/3000091410 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42100cd6c0 con 0x7f42100c9970 2023-07-08T12:38:35.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.265+0000 7f970511a700 1 --2- 172.21.15.12:0/4167204430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97000b1c70 0x7f97000ce580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f97000c6870 tx=0x7f96f000bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30e214dfd2455ff server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.265+0000 7f4214c85700 1 --2- 172.21.15.12:0/3000091410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42100c9970 0x7f42100cdec0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0091b0 tx=0x7f41fc0094c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f5b5a48afa68ac server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.265+0000 7f9704919700 1 -- 172.21.15.12:0/4167204430 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0012b80 con 0x7f97000b1c70 2023-07-08T12:38:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.266+0000 7f9704919700 1 -- 172.21.15.12:0/4167204430 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f000be90 con 0x7f97000b1c70 2023-07-08T12:38:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.266+0000 7f9704919700 1 -- 172.21.15.12:0/4167204430 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0004030 con 0x7f97000b1c70 2023-07-08T12:38:35.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.266+0000 7f420f7fe700 1 -- 172.21.15.12:0/3000091410 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc011830 con 0x7f42100c9970 2023-07-08T12:38:35.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.266+0000 7f420f7fe700 1 -- 172.21.15.12:0/3000091410 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc011990 con 0x7f42100c9970 2023-07-08T12:38:35.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.266+0000 7f420f7fe700 1 -- 172.21.15.12:0/3000091410 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc011d20 con 0x7f42100c9970 2023-07-08T12:38:35.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.266+0000 7f4216ee9700 1 -- 172.21.15.12:0/3000091410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42100c9970 msgr2=0x7f42100cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.266+0000 7f4216ee9700 1 --2- 172.21.15.12:0/3000091410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42100c9970 0x7f42100cdec0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0091b0 tx=0x7f41fc0094c0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.266+0000 7f9707b7f700 1 -- 172.21.15.12:0/4167204430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97000b1c70 msgr2=0x7f97000ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.266+0000 7f9707b7f700 1 --2- 172.21.15.12:0/4167204430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97000b1c70 0x7f97000ce580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f97000c6870 tx=0x7f96f000bdc0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.266+0000 7f4216ee9700 1 -- 172.21.15.12:0/3000091410 shutdown_connections 2023-07-08T12:38:35.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.266+0000 7f4216ee9700 1 --2- 172.21.15.12:0/3000091410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42100c9970 0x7f42100cdec0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0091b0 tx=0x7f41fc0094c0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7f4216ee9700 1 --2- 172.21.15.12:0/3000091410 >> v2:172.21.15.12:3300/0 conn(0x7f42100c9d40 0x7f42100cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7f4216ee9700 1 -- 172.21.15.12:0/3000091410 >> 172.21.15.12:0/3000091410 conn(0x7f421006abc0 msgr2=0x7f42100b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7fce40ff9700 1 -- 172.21.15.12:0/2769713395 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fce3c0145c0 con 0x7fce4406c550 2023-07-08T12:38:35.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7fce4a80b700 1 -- 172.21.15.12:0/2769713395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fce4406e640 con 0x7fce4406c550 2023-07-08T12:38:35.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7f9707b7f700 1 -- 172.21.15.12:0/4167204430 shutdown_connections 2023-07-08T12:38:35.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7f9707b7f700 1 --2- 172.21.15.12:0/4167204430 >> v2:172.21.15.160:3300/0 conn(0x7f970015d6c0 0x7f9700071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7f9707b7f700 1 --2- 172.21.15.12:0/4167204430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97000b1c70 0x7f97000ce580 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7f9707b7f700 1 -- 172.21.15.12:0/4167204430 >> 172.21.15.12:0/4167204430 conn(0x7f97000cc290 msgr2=0x7f97000cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7f9707b7f700 1 -- 172.21.15.12:0/4167204430 shutdown_connections 2023-07-08T12:38:35.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7f4216ee9700 1 -- 172.21.15.12:0/3000091410 shutdown_connections 2023-07-08T12:38:35.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7f9707b7f700 1 -- 172.21.15.12:0/4167204430 wait complete. 2023-07-08T12:38:35.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7f4216ee9700 1 -- 172.21.15.12:0/3000091410 wait complete. 2023-07-08T12:38:35.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.269+0000 7f4216ee9700 1 Processor -- start 2023-07-08T12:38:35.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.269+0000 7fb60ac49700 1 -- 172.21.15.12:0/3230281903 shutdown_connections 2023-07-08T12:38:35.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.269+0000 7fb60ac49700 1 -- 172.21.15.12:0/3230281903 wait complete. 2023-07-08T12:38:35.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.269+0000 7f9707b7f700 1 Processor -- start 2023-07-08T12:38:35.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7fce40ff9700 1 --2- 172.21.15.12:0/2769713395 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fce2c06ae60 0x7fce2c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7fce40ff9700 1 -- 172.21.15.12:0/2769713395 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fce3c089530 con 0x7fce4406c550 2023-07-08T12:38:35.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.267+0000 7fce437fe700 1 --2- 172.21.15.12:0/2769713395 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fce2c06ae60 0x7fce2c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.268+0000 7fce40ff9700 1 --2- 172.21.15.12:0/2769713395 >> [v2:172.21.15.12:6816/2950255941,v1:172.21.15.12:6817/2950255941] conn(0x7fce2c074a30 0x7fce2c076e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.268+0000 7fce40ff9700 1 -- 172.21.15.12:0/2769713395 --> [v2:172.21.15.12:6816/2950255941,v1:172.21.15.12:6817/2950255941] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce2c077530 con 0x7fce2c074a30 2023-07-08T12:38:35.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.268+0000 7fce40ff9700 1 -- 172.21.15.12:0/2769713395 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce3c054130 con 0x7fce4406c550 2023-07-08T12:38:35.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.268+0000 7fce48da8700 1 --2- 172.21.15.12:0/2769713395 >> [v2:172.21.15.12:6816/2950255941,v1:172.21.15.12:6817/2950255941] conn(0x7fce2c074a30 0x7fce2c076e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.268+0000 7fce437fe700 1 --2- 172.21.15.12:0/2769713395 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fce2c06ae60 0x7fce2c06d310 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce380098f0 tx=0x7fce38010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.268+0000 7fce48da8700 1 --2- 172.21.15.12:0/2769713395 >> [v2:172.21.15.12:6816/2950255941,v1:172.21.15.12:6817/2950255941] conn(0x7fce2c074a30 0x7fce2c076e60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7fb60ac49700 1 Processor -- start 2023-07-08T12:38:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.269+0000 7f4216ee9700 1 -- start start 2023-07-08T12:38:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.270+0000 7f4216ee9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42100c9d40 0x7f4210152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.270+0000 7f4216ee9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42100cb6e0 0x7f421015ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.270+0000 7f4216ee9700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42100b1c20 0x7f421015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.270+0000 7f4216ee9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42100c70c0 con 0x7f42100cb6e0 2023-07-08T12:38:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.270+0000 7f4216ee9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42100c6510 con 0x7f42100c9d40 2023-07-08T12:38:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.270+0000 7f4216ee9700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42101532e0 con 0x7f42100b1c20 2023-07-08T12:38:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.269+0000 7f9707b7f700 1 -- start start 2023-07-08T12:38:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f9707b7f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97000b1c70 0x7f970014db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f9707b7f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97000ce1b0 0x7f97000c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f9707b7f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f970015d6c0 0x7f97000c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f9707b7f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000b31f0 con 0x7f970015d6c0 2023-07-08T12:38:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f9707b7f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000b3490 con 0x7f97000ce1b0 2023-07-08T12:38:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f9707b7f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97000b3770 con 0x7f97000b1c70 2023-07-08T12:38:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f970611c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f970015d6c0 0x7f97000c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7fce40ff9700 1 -- 172.21.15.12:0/2769713395 <== osd.0 v2:172.21.15.12:6816/2950255941 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fce2c077530 con 0x7fce2c074a30 2023-07-08T12:38:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f970611c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f970015d6c0 0x7f97000c58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49686/0 (socket says 172.21.15.12:49686) 2023-07-08T12:38:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f970611c700 1 -- 172.21.15.12:0/2062225338 learned_addr learned my addr 172.21.15.12:0/2062225338 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f970511a700 1 --2- 172.21.15.12:0/2062225338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97000ce1b0 0x7f97000c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f4214c85700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42100c9d40 0x7f4210152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f4214c85700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42100c9d40 0x7f4210152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48422/0 (socket says 172.21.15.12:48422) 2023-07-08T12:38:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f4215486700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42100b1c20 0x7f421015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f4214c85700 1 -- 172.21.15.12:0/286735670 learned_addr learned my addr 172.21.15.12:0/286735670 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f4214c85700 1 -- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42100b1c20 msgr2=0x7f421015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f4214c85700 1 --2- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42100b1c20 0x7f421015b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f4214c85700 1 -- 172.21.15.12:0/286735670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42100cb6e0 msgr2=0x7f421015ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f4214c85700 1 --2- 172.21.15.12:0/286735670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42100cb6e0 0x7f421015ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f4214c85700 1 -- 172.21.15.12:0/286735670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc009d30 con 0x7f42100c9d40 2023-07-08T12:38:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7f4214c85700 1 --2- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42100c9d40 0x7f4210152da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f41fc000c00 tx=0x7f41fc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.272+0000 7f970511a700 1 -- 172.21.15.12:0/2062225338 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97000b1c70 msgr2=0x7f970014db60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-08T12:38:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.272+0000 7f970511a700 1 --2- 172.21.15.12:0/2062225338 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f97000b1c70 0x7f970014db60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.272+0000 7f970511a700 1 -- 172.21.15.12:0/2062225338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f970015d6c0 msgr2=0x7f97000c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.272+0000 7f970511a700 1 --2- 172.21.15.12:0/2062225338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f970015d6c0 0x7f97000c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.272+0000 7f970511a700 1 -- 172.21.15.12:0/2062225338 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f000b740 con 0x7f97000ce1b0 2023-07-08T12:38:35.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.272+0000 7f420d7fa700 1 -- 172.21.15.12:0/286735670 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0044d0 con 0x7f42100c9d40 2023-07-08T12:38:35.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.272+0000 7f420d7fa700 1 -- 172.21.15.12:0/286735670 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc01f070 con 0x7f42100c9d40 2023-07-08T12:38:35.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.272+0000 7f420d7fa700 1 -- 172.21.15.12:0/286735670 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0038e0 con 0x7f42100c9d40 2023-07-08T12:38:35.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.272+0000 7f970511a700 1 --2- 172.21.15.12:0/2062225338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97000ce1b0 0x7f97000c53a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f96f0019000 tx=0x7f96f0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.272+0000 7f4216ee9700 1 -- 172.21.15.12:0/286735670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42100c7550 con 0x7f42100c9d40 2023-07-08T12:38:35.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.272+0000 7f4216ee9700 1 -- 172.21.15.12:0/286735670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42100c6a70 con 0x7f42100c9d40 2023-07-08T12:38:35.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.271+0000 7fb60ac49700 1 -- start start 2023-07-08T12:38:35.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb60ac49700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6040b2b20 0x7fb6040c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb60ac49700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb60415f7f0 0x7fb604157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb60ac49700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb60415eb30 0x7fb6040c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb60ac49700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb604158220 con 0x7fb60415f7f0 2023-07-08T12:38:35.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb60ac49700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6040c80d0 con 0x7fb60415eb30 2023-07-08T12:38:35.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb60ac49700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6040c83b0 con 0x7fb6040b2b20 2023-07-08T12:38:35.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb6089e5700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6040b2b20 0x7fb6040c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb6089e5700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6040b2b20 0x7fb6040c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59308/0 (socket says 172.21.15.12:59308) 2023-07-08T12:38:35.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb6089e5700 1 -- 172.21.15.12:0/1734872187 learned_addr learned my addr 172.21.15.12:0/1734872187 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb603fff700 1 --2- 172.21.15.12:0/1734872187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb60415f7f0 0x7fb604157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb6089e5700 1 -- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb60415eb30 msgr2=0x7fb6040c9170 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-08T12:38:35.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb6089e5700 1 --2- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb60415eb30 0x7fb6040c9170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb6089e5700 1 -- 172.21.15.12:0/1734872187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb60415f7f0 msgr2=0x7fb604157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb6089e5700 1 --2- 172.21.15.12:0/1734872187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb60415f7f0 0x7fb604157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.273+0000 7fb6089e5700 1 -- 172.21.15.12:0/1734872187 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f4009550 con 0x7fb6040b2b20 2023-07-08T12:38:35.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.274+0000 7fb6089e5700 1 --2- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6040b2b20 0x7fb6040c7a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009fc0 tx=0x7fb5f800b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.275+0000 7fb6017fa700 1 -- 172.21.15.12:0/1734872187 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8033640 con 0x7fb6040b2b20 2023-07-08T12:38:35.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.275+0000 7fb6017fa700 1 -- 172.21.15.12:0/1734872187 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f80337c0 con 0x7fb6040b2b20 2023-07-08T12:38:35.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.275+0000 7fb6017fa700 1 -- 172.21.15.12:0/1734872187 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8033bb0 con 0x7fb6040b2b20 2023-07-08T12:38:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.275+0000 7fb60ac49700 1 -- 172.21.15.12:0/1734872187 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6040c8690 con 0x7fb6040b2b20 2023-07-08T12:38:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.275+0000 7fb60ac49700 1 -- 172.21.15.12:0/1734872187 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6040c8c50 con 0x7fb6040b2b20 2023-07-08T12:38:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.277+0000 7fb6017fa700 1 -- 172.21.15.12:0/1734872187 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb5f8033d10 con 0x7fb6040b2b20 2023-07-08T12:38:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.277+0000 7fb6017fa700 1 --2- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb5f006ad30 0x7fb5f006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.277+0000 7fb6017fa700 1 -- 172.21.15.12:0/1734872187 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb5f80ab520 con 0x7fb6040b2b20 2023-07-08T12:38:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.278+0000 7ff3b4ffd700 1 Processor -- start 2023-07-08T12:38:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.278+0000 7ff3b4ffd700 1 -- start start 2023-07-08T12:38:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.278+0000 7f96f67fc700 1 -- 172.21.15.12:0/2062225338 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0004240 con 0x7f97000ce1b0 2023-07-08T12:38:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.278+0000 7f96f67fc700 1 -- 172.21.15.12:0/2062225338 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f00043c0 con 0x7f97000ce1b0 2023-07-08T12:38:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.278+0000 7f9707b7f700 1 -- 172.21.15.12:0/2062225338 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97000b3a50 con 0x7f97000ce1b0 2023-07-08T12:38:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.278+0000 7f9707b7f700 1 -- 172.21.15.12:0/2062225338 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970014e0a0 con 0x7f97000ce1b0 2023-07-08T12:38:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.278+0000 7f96f67fc700 1 -- 172.21.15.12:0/2062225338 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f000d760 con 0x7f97000ce1b0 2023-07-08T12:38:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.279+0000 7f9707b7f700 1 -- 172.21.15.12:0/2062225338 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f96e8000fc0 con 0x7f97000ce1b0 2023-07-08T12:38:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.279+0000 7ff3b4ffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff3b00c9820 0x7ff3b0160a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.279+0000 7ff3b4ffd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3b00cb190 0x7ff3b00cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.279+0000 7ff3b4ffd700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b00ccb20 con 0x7ff3b00ad7c0 2023-07-08T12:38:35.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.279+0000 7ff3b4ffd700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b00cbb50 con 0x7ff3b00c9820 2023-07-08T12:38:35.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.279+0000 7ff3b4ffd700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00cc550 con 0x7ff3b00cb190 2023-07-08T12:38:35.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.280+0000 7ff3add9b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff3b00c9820 0x7ff3b0160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.280+0000 7ff3add9b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff3b00c9820 0x7ff3b0160a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49704/0 (socket says 172.21.15.12:49704) 2023-07-08T12:38:35.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.280+0000 7ff3add9b700 1 -- 172.21.15.12:0/1660162962 learned_addr learned my addr 172.21.15.12:0/1660162962 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.280+0000 7ff3add9b700 1 -- 172.21.15.12:0/1660162962 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3b00cb190 msgr2=0x7ff3b00cb610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-08T12:38:35.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.280+0000 7ff3add9b700 1 --2- 172.21.15.12:0/1660162962 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3b00cb190 0x7ff3b00cb610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.275+0000 7f4216ee9700 1 -- 172.21.15.12:0/286735670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42100cbb80 con 0x7f42100c9d40 2023-07-08T12:38:35.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.277+0000 7f420d7fa700 1 -- 172.21.15.12:0/286735670 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41fc003a40 con 0x7f42100c9d40 2023-07-08T12:38:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.277+0000 7f420d7fa700 1 --2- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f420006ae70 0x7f420006d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.278+0000 7f420d7fa700 1 -- 172.21.15.12:0/286735670 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f41fc021070 con 0x7f42100c9d40 2023-07-08T12:38:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.278+0000 7f420d7fa700 1 --2- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:6809/4149495008,v1:172.21.15.160:6811/4149495008] conn(0x7f4200074b40 0x7f4200076f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.278+0000 7f420d7fa700 1 -- 172.21.15.12:0/286735670 --> [v2:172.21.15.160:6809/4149495008,v1:172.21.15.160:6811/4149495008] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4200077640 con 0x7f4200074b40 2023-07-08T12:38:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.278+0000 7f420d7fa700 1 -- 172.21.15.12:0/286735670 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41fc08cd00 con 0x7f42100c9d40 2023-07-08T12:38:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.278+0000 7f420ffff700 1 --2- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f420006ae70 0x7f420006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.278+0000 7f4215486700 1 --2- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:6809/4149495008,v1:172.21.15.160:6811/4149495008] conn(0x7f4200074b40 0x7f4200076f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.280+0000 7f420ffff700 1 --2- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f420006ae70 0x7f420006d320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4204006b20 tx=0x7f4204006ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.280+0000 7f4215486700 1 --2- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:6809/4149495008,v1:172.21.15.160:6811/4149495008] conn(0x7f4200074b40 0x7f4200076f70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.280+0000 7ff3add9b700 1 -- 172.21.15.12:0/1660162962 >> v1:172.21.15.12:6789/0 conn(0x7ff3b00ad7c0 legacy=0x7ff3b00ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.280+0000 7ff3add9b700 1 -- 172.21.15.12:0/1660162962 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b00ccd70 con 0x7ff3b00c9820 2023-07-08T12:38:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.280+0000 7ff3add9b700 1 --2- 172.21.15.12:0/1660162962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3b00c9820 0x7ff3b0160a30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009920 tx=0x7ff3a4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a133848ff5bc7285 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.281+0000 7fd1950f4700 1 Processor -- start 2023-07-08T12:38:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.281+0000 7fb603fff700 1 --2- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb5f006ad30 0x7fb5f006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.281+0000 7ff3ad59a700 1 -- 172.21.15.12:0/1660162962 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400ba20 con 0x7ff3b00c9820 2023-07-08T12:38:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.281+0000 7ff3ad59a700 1 -- 172.21.15.12:0/1660162962 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a400bb80 con 0x7ff3b00c9820 2023-07-08T12:38:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.281+0000 7ff3ad59a700 1 -- 172.21.15.12:0/1660162962 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4009e70 con 0x7ff3b00c9820 2023-07-08T12:38:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.281+0000 7f96f67fc700 1 -- 172.21.15.12:0/2062225338 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f96f000b040 con 0x7f97000ce1b0 2023-07-08T12:38:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.281+0000 7f420d7fa700 1 -- 172.21.15.12:0/286735670 <== osd.7 v2:172.21.15.160:6809/4149495008 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4200077640 con 0x7f4200074b40 2023-07-08T12:38:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7f96f67fc700 1 --2- 172.21.15.12:0/2062225338 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f96e406ad30 0x7f96e406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/1660162962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3b00c9820 msgr2=0x7ff3b0160a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7ff3b4ffd700 1 --2- 172.21.15.12:0/1660162962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3b00c9820 0x7ff3b0160a30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009920 tx=0x7ff3a4009c30 comp rx=0 tx=0).stop 2023-07-08T12:38:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.281+0000 7fd1950f4700 1 -- start start 2023-07-08T12:38:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7fd1950f4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd1900c9970 0x7fd1900cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7fd1950f4700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd1900cb8b0 0x7fd190157d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7fd1950f4700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1900cce20 con 0x7fd1900c9d40 2023-07-08T12:38:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7fd1950f4700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1900cd100 con 0x7fd1900c9970 2023-07-08T12:38:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7fd1950f4700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1900cd3e0 con 0x7fd1900cb8b0 2023-07-08T12:38:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7f96f67fc700 1 -- 172.21.15.12:0/2062225338 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f96f008aa10 con 0x7f97000ce1b0 2023-07-08T12:38:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7f96f67fc700 1 --2- 172.21.15.12:0/2062225338 >> [v2:172.21.15.12:6824/1815509748,v1:172.21.15.12:6825/1815509748] conn(0x7f96e4074a00 0x7f96e4076e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7f96f67fc700 1 -- 172.21.15.12:0/2062225338 --> [v2:172.21.15.12:6824/1815509748,v1:172.21.15.12:6825/1815509748] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f96e4077500 con 0x7f96e4074a00 2023-07-08T12:38:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7f96f67fc700 1 -- 172.21.15.12:0/2062225338 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96f008f2c0 con 0x7f97000ce1b0 2023-07-08T12:38:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7fd18e59c700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd1900c9d40 0x7fd1900cb1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:46326/0 (socket says 172.21.15.12:46326) 2023-07-08T12:38:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7fd18e59c700 1 -- 172.21.15.12:0/1102562879 learned_addr learned my addr 172.21.15.12:0/1102562879 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7fd18f59e700 1 --2- 172.21.15.12:0/1102562879 >> v2:172.21.15.160:3300/0 conn(0x7fd1900cb8b0 0x7fd190157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7fd18ed9d700 1 --2- 172.21.15.12:0/1102562879 >> v2:172.21.15.12:3300/0 conn(0x7fd1900c9970 0x7fd1900cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7f970591b700 1 --2- 172.21.15.12:0/2062225338 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f96e406ad30 0x7f96e406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.282+0000 7f970611c700 1 --2- 172.21.15.12:0/2062225338 >> [v2:172.21.15.12:6824/1815509748,v1:172.21.15.12:6825/1815509748] conn(0x7f96e4074a00 0x7f96e4076e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.283+0000 7fd18dd9b700 1 -- 172.21.15.12:0/1102562879 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 935385573 0 0) 0x7fd1900cce20 con 0x7fd1900c9d40 2023-07-08T12:38:35.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.283+0000 7fd18dd9b700 1 -- 172.21.15.12:0/1102562879 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1780035b0 con 0x7fd1900c9d40 2023-07-08T12:38:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.283+0000 7fd18f59e700 1 -- 172.21.15.12:0/1102562879 >> v2:172.21.15.12:3300/0 conn(0x7fd1900c9970 msgr2=0x7fd1900cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.283+0000 7fd18f59e700 1 --2- 172.21.15.12:0/1102562879 >> v2:172.21.15.12:3300/0 conn(0x7fd1900c9970 0x7fd1900cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.283+0000 7fd18f59e700 1 -- 172.21.15.12:0/1102562879 >> v1:172.21.15.160:6789/0 conn(0x7fd1900c9d40 legacy=0x7fd1900cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.283+0000 7fd18f59e700 1 -- 172.21.15.12:0/1102562879 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1900cd6c0 con 0x7fd1900cb8b0 2023-07-08T12:38:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.283+0000 7fd18f59e700 1 --2- 172.21.15.12:0/1102562879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1900cb8b0 0x7fd190157d60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd18400bad0 tx=0x7fd18400bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e49446f67330635b server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.283+0000 7f970611c700 1 --2- 172.21.15.12:0/2062225338 >> [v2:172.21.15.12:6824/1815509748,v1:172.21.15.12:6825/1815509748] conn(0x7f96e4074a00 0x7f96e4076e30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.283+0000 7f970591b700 1 --2- 172.21.15.12:0/2062225338 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f96e406ad30 0x7f96e406d1e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96f8002220 tx=0x7f96f80017c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.283+0000 7fd18dd9b700 1 -- 172.21.15.12:0/1102562879 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184006d10 con 0x7fd1900cb8b0 2023-07-08T12:38:35.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.283+0000 7fd18dd9b700 1 -- 172.21.15.12:0/1102562879 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184006e70 con 0x7fd1900cb8b0 2023-07-08T12:38:35.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.283+0000 7fd18dd9b700 1 -- 172.21.15.12:0/1102562879 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18400dc20 con 0x7fd1900cb8b0 2023-07-08T12:38:35.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.284+0000 7fb60ac49700 1 --2- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:6823/2886140325,v1:172.21.15.160:6825/2886140325] conn(0x7fb5e40015e0 0x7fb5e4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.284+0000 7fb60ac49700 1 -- 172.21.15.12:0/1734872187 --> [v2:172.21.15.160:6823/2886140325,v1:172.21.15.160:6825/2886140325] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5e4006bc0 con 0x7fb5e40015e0 2023-07-08T12:38:35.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.284+0000 7fb6091e6700 1 --2- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:6823/2886140325,v1:172.21.15.160:6825/2886140325] conn(0x7fb5e40015e0 0x7fb5e4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.284+0000 7fb603fff700 1 --2- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb5f006ad30 0x7fb5f006d1e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb604158590 tx=0x7fb5fc00d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.284+0000 7f96f67fc700 1 -- 172.21.15.12:0/2062225338 <== osd.3 v2:172.21.15.12:6824/1815509748 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f96e4077500 con 0x7f96e4074a00 2023-07-08T12:38:35.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.284+0000 7fd1950f4700 1 -- 172.21.15.12:0/1102562879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1900cb8b0 msgr2=0x7fd190157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.284+0000 7fd1950f4700 1 --2- 172.21.15.12:0/1102562879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1900cb8b0 0x7fd190157d60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd18400bad0 tx=0x7fd18400bde0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.284+0000 7fd1950f4700 1 -- 172.21.15.12:0/1102562879 shutdown_connections 2023-07-08T12:38:35.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.284+0000 7fd1950f4700 1 --2- 172.21.15.12:0/1102562879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1900cb8b0 0x7fd190157d60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.284+0000 7fd1950f4700 1 --2- 172.21.15.12:0/1102562879 >> v2:172.21.15.12:3300/0 conn(0x7fd1900c9970 0x7fd1900cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.284+0000 7fd1950f4700 1 -- 172.21.15.12:0/1102562879 >> 172.21.15.12:0/1102562879 conn(0x7fd19006abc0 msgr2=0x7fd1900b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.284+0000 7fb6091e6700 1 --2- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:6823/2886140325,v1:172.21.15.160:6825/2886140325] conn(0x7fb5e40015e0 0x7fb5e4003a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.284+0000 7fd1950f4700 1 -- 172.21.15.12:0/1102562879 shutdown_connections 2023-07-08T12:38:35.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.285+0000 7fd1950f4700 1 -- 172.21.15.12:0/1102562879 wait complete. 2023-07-08T12:38:35.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.285+0000 7fb6017fa700 1 -- 172.21.15.12:0/1734872187 <== osd.5 v2:172.21.15.160:6823/2886140325 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb5e4006bc0 con 0x7fb5e40015e0 2023-07-08T12:38:35.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.286+0000 7f950daca700 1 Processor -- start 2023-07-08T12:38:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.286+0000 7fd1950f4700 1 Processor -- start 2023-07-08T12:38:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.287+0000 7faf0ceac700 1 -- 172.21.15.12:0/2479572126 --> [v2:172.21.15.12:6800/911521993,v1:172.21.15.12:6801/911521993] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faf080cd210 con 0x7faee8074930 2023-07-08T12:38:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.287+0000 7faef6ffd700 1 -- 172.21.15.12:0/2479572126 <== osd.1 v2:172.21.15.12:6800/911521993 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faef8006c60 con 0x7faee8074930 2023-07-08T12:38:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.288+0000 7faf0ceac700 1 -- 172.21.15.12:0/2479572126 >> [v2:172.21.15.12:6800/911521993,v1:172.21.15.12:6801/911521993] conn(0x7faee8074930 msgr2=0x7faee8076d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.288+0000 7faf0ceac700 1 --2- 172.21.15.12:0/2479572126 >> [v2:172.21.15.12:6800/911521993,v1:172.21.15.12:6801/911521993] conn(0x7faee8074930 0x7faee8076d60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.288+0000 7faf0ceac700 1 -- 172.21.15.12:0/2479572126 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faee806ace0 msgr2=0x7faee806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.288+0000 7faf0ceac700 1 --2- 172.21.15.12:0/2479572126 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faee806ace0 0x7faee806d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faefc0097b0 tx=0x7faefc006d20 comp rx=0 tx=0).stop 2023-07-08T12:38:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.288+0000 7faf0ceac700 1 -- 172.21.15.12:0/2479572126 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf080cda10 msgr2=0x7faf080b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.288+0000 7faf0ceac700 1 --2- 172.21.15.12:0/2479572126 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf080cda10 0x7faf080b20a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf0000ba50 tx=0x7faf0000bd60 comp rx=0 tx=0).stop 2023-07-08T12:38:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.288+0000 7faf06d9d700 1 -- 172.21.15.12:0/2479572126 reap_dead start 2023-07-08T12:38:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.288+0000 7faf0ceac700 1 -- 172.21.15.12:0/2479572126 shutdown_connections 2023-07-08T12:38:35.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.288+0000 7faf0ceac700 1 -- 172.21.15.12:0/2479572126 >> 172.21.15.12:0/2479572126 conn(0x7faf0806abc0 msgr2=0x7faf0806c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.288+0000 7faf0ceac700 1 -- 172.21.15.12:0/2479572126 shutdown_connections 2023-07-08T12:38:35.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.288+0000 7faf0ceac700 1 -- 172.21.15.12:0/2479572126 wait complete. 2023-07-08T12:38:35.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.295+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/1660162962 shutdown_connections 2023-07-08T12:38:35.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.295+0000 7ff3b4ffd700 1 --2- 172.21.15.12:0/1660162962 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3b00cb190 0x7ff3b00cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.295+0000 7ff3b4ffd700 1 --2- 172.21.15.12:0/1660162962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3b00c9820 0x7ff3b0160a30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.295+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/1660162962 >> 172.21.15.12:0/1660162962 conn(0x7ff3b006abc0 msgr2=0x7ff3b00b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.295+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/1660162962 shutdown_connections 2023-07-08T12:38:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.295+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/1660162962 wait complete. 2023-07-08T12:38:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.296+0000 7ff3b4ffd700 1 Processor -- start 2023-07-08T12:38:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.297+0000 7fd1950f4700 1 -- start start 2023-07-08T12:38:35.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.295+0000 7f950daca700 1 -- start start 2023-07-08T12:38:35.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.296+0000 7f950daca700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9508160930 0x7f9508070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.296+0000 7f950daca700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95080cc390 0x7f95080ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.296+0000 7f950daca700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95080ccb70 con 0x7f95080adce0 2023-07-08T12:38:35.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.296+0000 7f950daca700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95080ae160 con 0x7f9508160930 2023-07-08T12:38:35.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.296+0000 7f950daca700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9508165400 con 0x7f95080cc390 2023-07-08T12:38:35.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.296+0000 7f9506ffd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95080cc390 0x7f95080ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.296+0000 7f9506ffd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95080cc390 0x7f95080ad7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59340/0 (socket says 172.21.15.12:59340) 2023-07-08T12:38:35.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.296+0000 7f9506ffd700 1 -- 172.21.15.12:0/1219136945 learned_addr learned my addr 172.21.15.12:0/1219136945 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.296+0000 7f95077fe700 1 --2- 172.21.15.12:0/1219136945 >> v2:172.21.15.160:3300/0 conn(0x7f9508160930 0x7f9508070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.297+0000 7f9506ffd700 1 -- 172.21.15.12:0/1219136945 >> v2:172.21.15.160:3300/0 conn(0x7f9508160930 msgr2=0x7f9508070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.297+0000 7f9506ffd700 1 --2- 172.21.15.12:0/1219136945 >> v2:172.21.15.160:3300/0 conn(0x7f9508160930 0x7f9508070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.297+0000 7f9506ffd700 1 -- 172.21.15.12:0/1219136945 >> v1:172.21.15.160:6789/0 conn(0x7f95080adce0 legacy=0x7f9508069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.297+0000 7f9506ffd700 1 -- 172.21.15.12:0/1219136945 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9508164640 con 0x7f95080cc390 2023-07-08T12:38:35.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.297+0000 7ff3b4ffd700 1 -- start start 2023-07-08T12:38:35.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.297+0000 7f9506ffd700 1 --2- 172.21.15.12:0/1219136945 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95080cc390 0x7f95080ad7a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f94fc0098f0 tx=0x7f94fc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f795fc5f518c5a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3b4ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3b00ad7c0 0x7ff3b0158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3b4ffd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3b00c9820 0x7ff3b015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3b4ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3b00cb190 0x7ff3b015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3b4ffd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0159010 con 0x7ff3b00cb190 2023-07-08T12:38:35.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3b4ffd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b006cfa0 con 0x7ff3b00ad7c0 2023-07-08T12:38:35.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3b4ffd700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b006ccc0 con 0x7ff3b00c9820 2023-07-08T12:38:35.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3aed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3b00cb190 0x7ff3b015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3aed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3b00cb190 0x7ff3b015cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49726/0 (socket says 172.21.15.12:49726) 2023-07-08T12:38:35.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3aed9d700 1 -- 172.21.15.12:0/2190957521 learned_addr learned my addr 172.21.15.12:0/2190957521 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3ae59c700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3b00ad7c0 0x7ff3b0158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3add9b700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3b00c9820 0x7ff3b015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7f95067fc700 1 -- 172.21.15.12:0/1219136945 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc004230 con 0x7f95080cc390 2023-07-08T12:38:35.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7f95067fc700 1 -- 172.21.15.12:0/1219136945 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94fc004390 con 0x7f95080cc390 2023-07-08T12:38:35.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7f95067fc700 1 -- 172.21.15.12:0/1219136945 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc0036a0 con 0x7f95080cc390 2023-07-08T12:38:35.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.299+0000 7f950daca700 1 -- 172.21.15.12:0/1219136945 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95080cc390 msgr2=0x7f95080ad7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.299+0000 7f950daca700 1 --2- 172.21.15.12:0/1219136945 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95080cc390 0x7f95080ad7a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f94fc0098f0 tx=0x7f94fc009c00 comp rx=0 tx=0).stop 2023-07-08T12:38:35.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3ae59c700 1 -- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3b00c9820 msgr2=0x7ff3b015ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3ae59c700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3b00c9820 0x7ff3b015ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3ae59c700 1 -- 172.21.15.12:0/2190957521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3b00cb190 msgr2=0x7ff3b015cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3ae59c700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3b00cb190 0x7ff3b015cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.298+0000 7ff3ae59c700 1 -- 172.21.15.12:0/2190957521 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a4009550 con 0x7ff3b00ad7c0 2023-07-08T12:38:35.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.299+0000 7ff3ae59c700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3b00ad7c0 0x7ff3b0158ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0000f60 tx=0x7ff3a000c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7fd1950f4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1900c9970 0x7fd1901562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7fd1950f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1900c9d40 0x7fd1900b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7fd1950f4700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1900cb8b0 0x7fd1900c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7fd1950f4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900b1b90 con 0x7fd1900c9d40 2023-07-08T12:38:35.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7fd1950f4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900b1e70 con 0x7fd1900c9970 2023-07-08T12:38:35.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7fd1950f4700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900b2150 con 0x7fd1900cb8b0 2023-07-08T12:38:35.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7fd18e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1900c9d40 0x7fd1900b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7fd18e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1900c9d40 0x7fd1900b2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49734/0 (socket says 172.21.15.12:49734) 2023-07-08T12:38:35.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7fd18e59c700 1 -- 172.21.15.12:0/4114294765 learned_addr learned my addr 172.21.15.12:0/4114294765 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7fd18f59e700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1900cb8b0 0x7fd1900c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7fd18ed9d700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1900c9970 0x7fd1901562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.301+0000 7fd18e59c700 1 -- 172.21.15.12:0/4114294765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1900cb8b0 msgr2=0x7fd1900c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.301+0000 7fd18e59c700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1900cb8b0 0x7fd1900c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.301+0000 7fd18e59c700 1 -- 172.21.15.12:0/4114294765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1900c9970 msgr2=0x7fd1901562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.301+0000 7fd18e59c700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1900c9970 0x7fd1901562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.301+0000 7fd18e59c700 1 -- 172.21.15.12:0/4114294765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18400b730 con 0x7fd1900c9d40 2023-07-08T12:38:35.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.301+0000 7fd18ed9d700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1900c9970 0x7fd1901562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:35.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.301+0000 7fd18e59c700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1900c9d40 0x7fd1900b2c30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd18000c960 tx=0x7fd18000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.335 INFO:teuthology.orchestra.run.smithi012.stdout:55834574859 2023-07-08T12:38:35.336 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-08T12:38:35.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7ff39effd700 1 -- 172.21.15.12:0/2190957521 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00052d0 con 0x7ff3b00ad7c0 2023-07-08T12:38:35.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7ff39effd700 1 -- 172.21.15.12:0/2190957521 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a0005430 con 0x7ff3b00ad7c0 2023-07-08T12:38:35.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.300+0000 7ff39effd700 1 -- 172.21.15.12:0/2190957521 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00038c0 con 0x7ff3b00ad7c0 2023-07-08T12:38:35.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.302+0000 7f950daca700 1 -- 172.21.15.12:0/1219136945 shutdown_connections 2023-07-08T12:38:35.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.302+0000 7f950daca700 1 --2- 172.21.15.12:0/1219136945 >> v2:172.21.15.160:3300/0 conn(0x7f9508160930 0x7f9508070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.302+0000 7f950daca700 1 --2- 172.21.15.12:0/1219136945 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95080cc390 0x7f95080ad7a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.302+0000 7f950daca700 1 -- 172.21.15.12:0/1219136945 >> 172.21.15.12:0/1219136945 conn(0x7f95080b0390 msgr2=0x7f95080b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.302+0000 7fd1777fe700 1 -- 172.21.15.12:0/4114294765 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180007970 con 0x7fd1900c9d40 2023-07-08T12:38:35.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.302+0000 7fd1777fe700 1 -- 172.21.15.12:0/4114294765 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd180007ad0 con 0x7fd1900c9d40 2023-07-08T12:38:35.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.302+0000 7fd1777fe700 1 -- 172.21.15.12:0/4114294765 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180007e60 con 0x7fd1900c9d40 2023-07-08T12:38:35.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.303+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/2190957521 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b015e7e0 con 0x7ff3b00ad7c0 2023-07-08T12:38:35.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.303+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/2190957521 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b015eda0 con 0x7ff3b00ad7c0 2023-07-08T12:38:35.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.302+0000 7f950daca700 1 -- 172.21.15.12:0/1219136945 shutdown_connections 2023-07-08T12:38:35.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.304+0000 7f950daca700 1 -- 172.21.15.12:0/1219136945 wait complete. 2023-07-08T12:38:35.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.304+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/2190957521 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3b00cd330 con 0x7ff3b00ad7c0 2023-07-08T12:38:35.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.304+0000 7fd1950f4700 1 -- 172.21.15.12:0/4114294765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1900b2430 con 0x7fd1900c9d40 2023-07-08T12:38:35.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.304+0000 7fd1950f4700 1 -- 172.21.15.12:0/4114294765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1900b3170 con 0x7fd1900c9d40 2023-07-08T12:38:35.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.306+0000 7fd1950f4700 1 -- 172.21.15.12:0/4114294765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd1900cd9a0 con 0x7fd1900c9d40 2023-07-08T12:38:35.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.306+0000 7fce4a80b700 1 -- 172.21.15.12:0/2769713395 --> [v2:172.21.15.12:6816/2950255941,v1:172.21.15.12:6817/2950255941] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fce4415df70 con 0x7fce2c074a30 2023-07-08T12:38:35.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.306+0000 7f950daca700 1 Processor -- start 2023-07-08T12:38:35.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.306+0000 7ff39effd700 1 -- 172.21.15.12:0/2190957521 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff3a0003c30 con 0x7ff3b00ad7c0 2023-07-08T12:38:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.306+0000 7ff39effd700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff39806aec0 0x7ff39806d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.306+0000 7ff39effd700 1 -- 172.21.15.12:0/2190957521 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff3a00892e0 con 0x7ff3b00ad7c0 2023-07-08T12:38:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.307+0000 7ff39effd700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:6808/1968203867,v1:172.21.15.160:6810/1968203867] conn(0x7ff398074b90 0x7ff398076fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.307+0000 7ff39effd700 1 -- 172.21.15.12:0/2190957521 --> [v2:172.21.15.160:6808/1968203867,v1:172.21.15.160:6810/1968203867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff398077690 con 0x7ff398074b90 2023-07-08T12:38:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.307+0000 7ff39effd700 1 -- 172.21.15.12:0/2190957521 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3a0089690 con 0x7ff3b00ad7c0 2023-07-08T12:38:35.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.307+0000 7fce40ff9700 1 -- 172.21.15.12:0/2769713395 <== osd.0 v2:172.21.15.12:6816/2950255941 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fce34009280 con 0x7fce2c074a30 2023-07-08T12:38:35.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.307+0000 7fd1777fe700 1 -- 172.21.15.12:0/4114294765 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd180016420 con 0x7fd1900c9d40 2023-07-08T12:38:35.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.307+0000 7fce4a80b700 1 -- 172.21.15.12:0/2769713395 >> [v2:172.21.15.12:6816/2950255941,v1:172.21.15.12:6817/2950255941] conn(0x7fce2c074a30 msgr2=0x7fce2c076e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7fce4a80b700 1 --2- 172.21.15.12:0/2769713395 >> [v2:172.21.15.12:6816/2950255941,v1:172.21.15.12:6817/2950255941] conn(0x7fce2c074a30 0x7fce2c076e60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.307+0000 7f950daca700 1 -- start start 2023-07-08T12:38:35.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7fd1777fe700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd17c06ace0 0x7fd17c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7ff3aed9d700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:6808/1968203867,v1:172.21.15.160:6810/1968203867] conn(0x7ff398074b90 0x7ff398076fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7f950daca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95080adce0 0x7f9508161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7f950daca700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95080cc390 0x7f9508151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7f950daca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9508160930 0x7f9508150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7f950daca700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95080c3320 con 0x7f9508160930 2023-07-08T12:38:35.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7f950daca700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9508152000 con 0x7f95080adce0 2023-07-08T12:38:35.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7f950daca700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95080c3530 con 0x7f95080cc390 2023-07-08T12:38:35.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7fd1777fe700 1 -- 172.21.15.12:0/4114294765 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd180012070 con 0x7fd1900c9d40 2023-07-08T12:38:35.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7f95077fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95080adce0 0x7f9508161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7f95077fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95080adce0 0x7f9508161010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48486/0 (socket says 172.21.15.12:48486) 2023-07-08T12:38:35.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7f95077fe700 1 -- 172.21.15.12:0/2944174705 learned_addr learned my addr 172.21.15.12:0/2944174705 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7fd1777fe700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.12:6808/720039673,v1:172.21.15.12:6809/720039673] conn(0x7fd17c074930 0x7fd17c076d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7fd1777fe700 1 -- 172.21.15.12:0/4114294765 --> [v2:172.21.15.12:6808/720039673,v1:172.21.15.12:6809/720039673] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd17c077430 con 0x7fd17c074930 2023-07-08T12:38:35.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7fd1777fe700 1 -- 172.21.15.12:0/4114294765 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd18008a780 con 0x7fd1900c9d40 2023-07-08T12:38:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7ff3add9b700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff39806aec0 0x7ff39806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7f9507fff700 1 --2- 172.21.15.12:0/2944174705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9508160930 0x7f9508150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7fd18f59e700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.12:6808/720039673,v1:172.21.15.12:6809/720039673] conn(0x7fd17c074930 0x7fd17c076d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.308+0000 7fd18ed9d700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd17c06ace0 0x7fd17c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7ff3add9b700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff39806aec0 0x7ff39806d370 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40098f0 tx=0x7ff3a4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7f95077fe700 1 -- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95080cc390 msgr2=0x7f9508151990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7f95077fe700 1 --2- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95080cc390 0x7f9508151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7f95077fe700 1 -- 172.21.15.12:0/2944174705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9508160930 msgr2=0x7f9508150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7f95077fe700 1 --2- 172.21.15.12:0/2944174705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9508160930 0x7f9508150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7f95077fe700 1 -- 172.21.15.12:0/2944174705 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc009550 con 0x7f95080adce0 2023-07-08T12:38:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7f95077fe700 1 --2- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95080adce0 0x7f9508161010 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94f800e810 tx=0x7f94f800eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7fd18f59e700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.12:6808/720039673,v1:172.21.15.12:6809/720039673] conn(0x7fd17c074930 0x7fd17c076d60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7f94f7fff700 1 -- 172.21.15.12:0/2944174705 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f8013610 con 0x7f95080adce0 2023-07-08T12:38:35.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7ff3aed9d700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:6808/1968203867,v1:172.21.15.160:6810/1968203867] conn(0x7ff398074b90 0x7ff398076fc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7f94f7fff700 1 -- 172.21.15.12:0/2944174705 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f8013770 con 0x7f95080adce0 2023-07-08T12:38:35.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7f94f7fff700 1 -- 172.21.15.12:0/2944174705 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f8013b20 con 0x7f95080adce0 2023-07-08T12:38:35.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7f950daca700 1 -- 172.21.15.12:0/2944174705 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95080c3810 con 0x7f95080adce0 2023-07-08T12:38:35.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.309+0000 7f950daca700 1 -- 172.21.15.12:0/2944174705 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9508151380 con 0x7f95080adce0 2023-07-08T12:38:35.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.310+0000 7fd18ed9d700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd17c06ace0 0x7fd17c06d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd1880074c0 tx=0x7fd18800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.310+0000 7fce4a80b700 1 -- 172.21.15.12:0/2769713395 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fce2c06ae60 msgr2=0x7fce2c06d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.310+0000 7fce4a80b700 1 --2- 172.21.15.12:0/2769713395 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fce2c06ae60 0x7fce2c06d310 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce380098f0 tx=0x7fce38010000 comp rx=0 tx=0).stop 2023-07-08T12:38:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.310+0000 7fce4a80b700 1 -- 172.21.15.12:0/2769713395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce4406c550 msgr2=0x7fce4415a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.310+0000 7fce4a80b700 1 --2- 172.21.15.12:0/2769713395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce4406c550 0x7fce4415a760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00ba50 tx=0x7fce3c00be10 comp rx=0 tx=0).stop 2023-07-08T12:38:35.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.310+0000 7fd1777fe700 1 -- 172.21.15.12:0/4114294765 <== osd.2 v2:172.21.15.12:6808/720039673 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd17c077430 con 0x7fd17c074930 2023-07-08T12:38:35.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.310+0000 7fce48da8700 1 -- 172.21.15.12:0/2769713395 reap_dead start 2023-07-08T12:38:35.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.310+0000 7f950daca700 1 -- 172.21.15.12:0/2944174705 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94e4000fc0 con 0x7f95080adce0 2023-07-08T12:38:35.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.311+0000 7fce4a80b700 1 -- 172.21.15.12:0/2769713395 shutdown_connections 2023-07-08T12:38:35.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.311+0000 7fce4a80b700 1 -- 172.21.15.12:0/2769713395 >> 172.21.15.12:0/2769713395 conn(0x7fce4406abc0 msgr2=0x7fce440cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.311+0000 7ff39effd700 1 -- 172.21.15.12:0/2190957521 <== osd.4 v2:172.21.15.160:6808/1968203867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff398077690 con 0x7ff398074b90 2023-07-08T12:38:35.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.311+0000 7fce4a80b700 1 -- 172.21.15.12:0/2769713395 shutdown_connections 2023-07-08T12:38:35.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.311+0000 7fce4a80b700 1 -- 172.21.15.12:0/2769713395 wait complete. 2023-07-08T12:38:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.311+0000 7f94f7fff700 1 -- 172.21.15.12:0/2944174705 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f94f8013c80 con 0x7f95080adce0 2023-07-08T12:38:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.312+0000 7f94f7fff700 1 --2- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f94ec06ad00 0x7f94ec06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.312+0000 7f94f7fff700 1 -- 172.21.15.12:0/2944174705 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f94f8089510 con 0x7f95080adce0 2023-07-08T12:38:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.312+0000 7f9506ffd700 1 --2- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f94ec06ad00 0x7f94ec06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.312+0000 7f94f7fff700 1 --2- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:6800/2675224151,v1:172.21.15.160:6801/2675224151] conn(0x7f94ec0749d0 0x7f94ec076e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.312+0000 7f94f7fff700 1 -- 172.21.15.12:0/2944174705 --> [v2:172.21.15.160:6800/2675224151,v1:172.21.15.160:6801/2675224151] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94ec0774d0 con 0x7f94ec0749d0 2023-07-08T12:38:35.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.312+0000 7f94f7fff700 1 -- 172.21.15.12:0/2944174705 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94f8089960 con 0x7f95080adce0 2023-07-08T12:38:35.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.313+0000 7f9507fff700 1 --2- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:6800/2675224151,v1:172.21.15.160:6801/2675224151] conn(0x7f94ec0749d0 0x7f94ec076e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.313+0000 7f9506ffd700 1 --2- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f94ec06ad00 0x7f94ec06d1b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f95080aee10 tx=0x7f94fc010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.313+0000 7f9507fff700 1 --2- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:6800/2675224151,v1:172.21.15.160:6801/2675224151] conn(0x7f94ec0749d0 0x7f94ec076e00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.314+0000 7f4216ee9700 1 -- 172.21.15.12:0/286735670 --> [v2:172.21.15.160:6809/4149495008,v1:172.21.15.160:6811/4149495008] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f42100cd9a0 con 0x7f4200074b40 2023-07-08T12:38:35.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.314+0000 7f94f7fff700 1 -- 172.21.15.12:0/2944174705 <== osd.6 v2:172.21.15.160:6800/2675224151 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f94ec0774d0 con 0x7f94ec0749d0 2023-07-08T12:38:35.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.314+0000 7f420d7fa700 1 -- 172.21.15.12:0/286735670 <== osd.7 v2:172.21.15.160:6809/4149495008 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4208001ec0 con 0x7f4200074b40 2023-07-08T12:38:35.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.315+0000 7f4216ee9700 1 -- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:6809/4149495008,v1:172.21.15.160:6811/4149495008] conn(0x7f4200074b40 msgr2=0x7f4200076f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.315+0000 7f4216ee9700 1 --2- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:6809/4149495008,v1:172.21.15.160:6811/4149495008] conn(0x7f4200074b40 0x7f4200076f70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.315+0000 7f4216ee9700 1 -- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f420006ae70 msgr2=0x7f420006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.315+0000 7f4216ee9700 1 --2- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f420006ae70 0x7f420006d320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4204006b20 tx=0x7f4204006ab0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.315+0000 7f4216ee9700 1 -- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42100c9d40 msgr2=0x7f4210152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.315+0000 7f4216ee9700 1 --2- 172.21.15.12:0/286735670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42100c9d40 0x7f4210152da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f41fc000c00 tx=0x7f41fc004750 comp rx=0 tx=0).stop 2023-07-08T12:38:35.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.315+0000 7f4215486700 1 -- 172.21.15.12:0/286735670 reap_dead start 2023-07-08T12:38:35.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.316+0000 7f4216ee9700 1 -- 172.21.15.12:0/286735670 shutdown_connections 2023-07-08T12:38:35.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.316+0000 7f4216ee9700 1 -- 172.21.15.12:0/286735670 >> 172.21.15.12:0/286735670 conn(0x7f421006abc0 msgr2=0x7f42100c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.317+0000 7f9707b7f700 1 -- 172.21.15.12:0/2062225338 --> [v2:172.21.15.12:6824/1815509748,v1:172.21.15.12:6825/1815509748] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f96e8002d40 con 0x7f96e4074a00 2023-07-08T12:38:35.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.317+0000 7fb60ac49700 1 -- 172.21.15.12:0/1734872187 --> [v2:172.21.15.160:6823/2886140325,v1:172.21.15.160:6825/2886140325] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb5e4005c40 con 0x7fb5e40015e0 2023-07-08T12:38:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.317+0000 7f96f67fc700 1 -- 172.21.15.12:0/2062225338 <== osd.3 v2:172.21.15.12:6824/1815509748 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f96e8002d40 con 0x7f96e4074a00 2023-07-08T12:38:35.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.317+0000 7fb6017fa700 1 -- 172.21.15.12:0/1734872187 <== osd.5 v2:172.21.15.160:6823/2886140325 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb5e4005c40 con 0x7fb5e40015e0 2023-07-08T12:38:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7fb60ac49700 1 -- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:6823/2886140325,v1:172.21.15.160:6825/2886140325] conn(0x7fb5e40015e0 msgr2=0x7fb5e4003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7fb60ac49700 1 --2- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:6823/2886140325,v1:172.21.15.160:6825/2886140325] conn(0x7fb5e40015e0 0x7fb5e4003a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7fb60ac49700 1 -- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb5f006ad30 msgr2=0x7fb5f006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7fb60ac49700 1 --2- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb5f006ad30 0x7fb5f006d1e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb604158590 tx=0x7fb5fc00d040 comp rx=0 tx=0).stop 2023-07-08T12:38:35.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7fb60ac49700 1 -- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6040b2b20 msgr2=0x7fb6040c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7fb60ac49700 1 --2- 172.21.15.12:0/1734872187 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6040b2b20 0x7fb6040c7a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009fc0 tx=0x7fb5f800b830 comp rx=0 tx=0).stop 2023-07-08T12:38:35.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7fb6091e6700 1 -- 172.21.15.12:0/1734872187 reap_dead start 2023-07-08T12:38:35.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7fb60ac49700 1 -- 172.21.15.12:0/1734872187 shutdown_connections 2023-07-08T12:38:35.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7fb60ac49700 1 -- 172.21.15.12:0/1734872187 >> 172.21.15.12:0/1734872187 conn(0x7fb60406ab60 msgr2=0x7fb6040cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7f9707b7f700 1 -- 172.21.15.12:0/2062225338 >> [v2:172.21.15.12:6824/1815509748,v1:172.21.15.12:6825/1815509748] conn(0x7f96e4074a00 msgr2=0x7f96e4076e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7f9707b7f700 1 --2- 172.21.15.12:0/2062225338 >> [v2:172.21.15.12:6824/1815509748,v1:172.21.15.12:6825/1815509748] conn(0x7f96e4074a00 0x7f96e4076e30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7f9707b7f700 1 -- 172.21.15.12:0/2062225338 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f96e406ad30 msgr2=0x7f96e406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7f9707b7f700 1 --2- 172.21.15.12:0/2062225338 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f96e406ad30 0x7f96e406d1e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96f8002220 tx=0x7f96f80017c0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7f9707b7f700 1 -- 172.21.15.12:0/2062225338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97000ce1b0 msgr2=0x7f97000c53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7f9707b7f700 1 --2- 172.21.15.12:0/2062225338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97000ce1b0 0x7f97000c53a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f96f0019000 tx=0x7f96f0004750 comp rx=0 tx=0).stop 2023-07-08T12:38:35.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7f970611c700 1 -- 172.21.15.12:0/2062225338 reap_dead start 2023-07-08T12:38:35.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7fb60ac49700 1 -- 172.21.15.12:0/1734872187 shutdown_connections 2023-07-08T12:38:35.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.318+0000 7fb60ac49700 1 -- 172.21.15.12:0/1734872187 wait complete. 2023-07-08T12:38:35.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.319+0000 7f4216ee9700 1 -- 172.21.15.12:0/286735670 shutdown_connections 2023-07-08T12:38:35.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.319+0000 7f4216ee9700 1 -- 172.21.15.12:0/286735670 wait complete. 2023-07-08T12:38:35.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.325+0000 7f9707b7f700 1 -- 172.21.15.12:0/2062225338 shutdown_connections 2023-07-08T12:38:35.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.325+0000 7f9707b7f700 1 -- 172.21.15.12:0/2062225338 >> 172.21.15.12:0/2062225338 conn(0x7f97000cc290 msgr2=0x7f97000af370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.326+0000 7f9707b7f700 1 -- 172.21.15.12:0/2062225338 shutdown_connections 2023-07-08T12:38:35.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.326+0000 7f9707b7f700 1 -- 172.21.15.12:0/2062225338 wait complete. 2023-07-08T12:38:35.367 INFO:teuthology.orchestra.run.smithi012.stdout:55834574859 2023-07-08T12:38:35.368 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-08T12:38:35.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.335+0000 7f950daca700 1 -- 172.21.15.12:0/2944174705 --> [v2:172.21.15.160:6800/2675224151,v1:172.21.15.160:6801/2675224151] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f94e4002d40 con 0x7f94ec0749d0 2023-07-08T12:38:35.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.336+0000 7f94f7fff700 1 -- 172.21.15.12:0/2944174705 <== osd.6 v2:172.21.15.160:6800/2675224151 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f94e4002d40 con 0x7f94ec0749d0 2023-07-08T12:38:35.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.336+0000 7f950daca700 1 -- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:6800/2675224151,v1:172.21.15.160:6801/2675224151] conn(0x7f94ec0749d0 msgr2=0x7f94ec076e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.336+0000 7f950daca700 1 --2- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:6800/2675224151,v1:172.21.15.160:6801/2675224151] conn(0x7f94ec0749d0 0x7f94ec076e00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.336+0000 7f950daca700 1 -- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f94ec06ad00 msgr2=0x7f94ec06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.336+0000 7f950daca700 1 --2- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f94ec06ad00 0x7f94ec06d1b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f95080aee10 tx=0x7f94fc010000 comp rx=0 tx=0).stop 2023-07-08T12:38:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.336+0000 7f950daca700 1 -- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95080adce0 msgr2=0x7f9508161010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.336+0000 7f950daca700 1 --2- 172.21.15.12:0/2944174705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95080adce0 0x7f9508161010 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94f800e810 tx=0x7f94f800eb20 comp rx=0 tx=0).stop 2023-07-08T12:38:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.336+0000 7f9507fff700 1 -- 172.21.15.12:0/2944174705 reap_dead start 2023-07-08T12:38:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.336+0000 7f950daca700 1 -- 172.21.15.12:0/2944174705 shutdown_connections 2023-07-08T12:38:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.336+0000 7f950daca700 1 -- 172.21.15.12:0/2944174705 >> 172.21.15.12:0/2944174705 conn(0x7f95080b0390 msgr2=0x7f95080b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.337+0000 7f950daca700 1 -- 172.21.15.12:0/2944174705 shutdown_connections 2023-07-08T12:38:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.337+0000 7f950daca700 1 -- 172.21.15.12:0/2944174705 wait complete. 2023-07-08T12:38:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.338+0000 7fd1950f4700 1 -- 172.21.15.12:0/4114294765 --> [v2:172.21.15.12:6808/720039673,v1:172.21.15.12:6809/720039673] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd19004ea60 con 0x7fd17c074930 2023-07-08T12:38:35.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.338+0000 7fd1777fe700 1 -- 172.21.15.12:0/4114294765 <== osd.2 v2:172.21.15.12:6808/720039673 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd19004ea60 con 0x7fd17c074930 2023-07-08T12:38:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.339+0000 7fd1950f4700 1 -- 172.21.15.12:0/4114294765 >> [v2:172.21.15.12:6808/720039673,v1:172.21.15.12:6809/720039673] conn(0x7fd17c074930 msgr2=0x7fd17c076d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.339+0000 7fd1950f4700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.12:6808/720039673,v1:172.21.15.12:6809/720039673] conn(0x7fd17c074930 0x7fd17c076d60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.339+0000 7fd1950f4700 1 -- 172.21.15.12:0/4114294765 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd17c06ace0 msgr2=0x7fd17c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.339+0000 7fd1950f4700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd17c06ace0 0x7fd17c06d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd1880074c0 tx=0x7fd18800b040 comp rx=0 tx=0).stop 2023-07-08T12:38:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.339+0000 7fd1950f4700 1 -- 172.21.15.12:0/4114294765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1900c9d40 msgr2=0x7fd1900b2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.339+0000 7fd1950f4700 1 --2- 172.21.15.12:0/4114294765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1900c9d40 0x7fd1900b2c30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd18000c960 tx=0x7fd18000cc70 comp rx=0 tx=0).stop 2023-07-08T12:38:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.339+0000 7fd18f59e700 1 -- 172.21.15.12:0/4114294765 reap_dead start 2023-07-08T12:38:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.339+0000 7fd1950f4700 1 -- 172.21.15.12:0/4114294765 shutdown_connections 2023-07-08T12:38:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.339+0000 7fd1950f4700 1 -- 172.21.15.12:0/4114294765 >> 172.21.15.12:0/4114294765 conn(0x7fd19006abc0 msgr2=0x7fd19006e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.339+0000 7fd1950f4700 1 -- 172.21.15.12:0/4114294765 shutdown_connections 2023-07-08T12:38:35.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.339+0000 7fd1950f4700 1 -- 172.21.15.12:0/4114294765 wait complete. 2023-07-08T12:38:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.339+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/2190957521 --> [v2:172.21.15.160:6808/1968203867,v1:172.21.15.160:6810/1968203867] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff3b00cd050 con 0x7ff398074b90 2023-07-08T12:38:35.374 INFO:teuthology.orchestra.run.smithi012.stdout:55834574860 2023-07-08T12:38:35.375 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-08T12:38:35.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.340+0000 7ff39effd700 1 -- 172.21.15.12:0/2190957521 <== osd.4 v2:172.21.15.160:6808/1968203867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff3a800f040 con 0x7ff398074b90 2023-07-08T12:38:35.376 INFO:teuthology.orchestra.run.smithi012.stdout:55834574861 2023-07-08T12:38:35.376 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-08T12:38:35.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.341+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:6808/1968203867,v1:172.21.15.160:6810/1968203867] conn(0x7ff398074b90 msgr2=0x7ff398076fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.341+0000 7ff3b4ffd700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:6808/1968203867,v1:172.21.15.160:6810/1968203867] conn(0x7ff398074b90 0x7ff398076fc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.341+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff39806aec0 msgr2=0x7ff39806d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.341+0000 7ff3b4ffd700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff39806aec0 0x7ff39806d370 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40098f0 tx=0x7ff3a4008030 comp rx=0 tx=0).stop 2023-07-08T12:38:35.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.341+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3b00ad7c0 msgr2=0x7ff3b0158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.341+0000 7ff3b4ffd700 1 --2- 172.21.15.12:0/2190957521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3b00ad7c0 0x7ff3b0158ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0000f60 tx=0x7ff3a000c6e0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.341+0000 7ff3aed9d700 1 -- 172.21.15.12:0/2190957521 reap_dead start 2023-07-08T12:38:35.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.345+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/2190957521 shutdown_connections 2023-07-08T12:38:35.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.345+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/2190957521 >> 172.21.15.12:0/2190957521 conn(0x7ff3b006abc0 msgr2=0x7ff3b015bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.345+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/2190957521 shutdown_connections 2023-07-08T12:38:35.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.345+0000 7ff3b4ffd700 1 -- 172.21.15.12:0/2190957521 wait complete. 2023-07-08T12:38:35.379 INFO:teuthology.orchestra.run.smithi012.stdout:55834574860 2023-07-08T12:38:35.380 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-08T12:38:35.381 INFO:teuthology.orchestra.run.smithi012.stdout:55834574860 2023-07-08T12:38:35.381 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-08T12:38:35.382 INFO:teuthology.orchestra.run.smithi012.stdout:55834574860 2023-07-08T12:38:35.382 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-08T12:38:35.383 INFO:teuthology.orchestra.run.smithi012.stdout:55834574860 2023-07-08T12:38:35.383 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-08T12:38:35.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.606+0000 7fae2d21a700 1 Processor -- start 2023-07-08T12:38:35.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.606+0000 7fae2d21a700 1 -- start start 2023-07-08T12:38:35.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.607+0000 7fae2d21a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fae280c9970 0x7fae280cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.607+0000 7fae2d21a700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae280cce20 con 0x7fae280cb8b0 2023-07-08T12:38:35.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.607+0000 7fae2d21a700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae280cd100 con 0x7fae280c9d40 2023-07-08T12:38:35.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.608+0000 7fae2d21a700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae280cd3e0 con 0x7fae280c9970 2023-07-08T12:38:35.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.608+0000 7fae2659c700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fae280c9d40 0x7fae280cb1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:46374/0 (socket says 172.21.15.12:46374) 2023-07-08T12:38:35.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.608+0000 7fae2659c700 1 -- 172.21.15.12:0/2834578862 learned_addr learned my addr 172.21.15.12:0/2834578862 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:35.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae26d9d700 1 --2- 172.21.15.12:0/2834578862 >> v2:172.21.15.12:3300/0 conn(0x7fae280c9970 0x7fae280cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124782417 0 0) 0x7fae280cce20 con 0x7fae280cb8b0 2023-07-08T12:38:35.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae100034d0 con 0x7fae280cb8b0 2023-07-08T12:38:35.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1904701047 0 0) 0x7fae100034d0 con 0x7fae280cb8b0 2023-07-08T12:38:35.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 >> v2:172.21.15.12:3300/0 conn(0x7fae280c9970 msgr2=0x7fae280cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 --2- 172.21.15.12:0/2834578862 >> v2:172.21.15.12:3300/0 conn(0x7fae280c9970 0x7fae280cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 >> v1:172.21.15.160:6789/0 conn(0x7fae280c9d40 legacy=0x7fae280cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae280cd6c0 con 0x7fae280cb8b0 2023-07-08T12:38:35.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7fae18001670 con 0x7fae280cb8b0 2023-07-08T12:38:35.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 >> v1:172.21.15.12:6789/0 conn(0x7fae280cb8b0 legacy=0x7fae28157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 --2- 172.21.15.12:0/2834578862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae10003af0 0x7fae10003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 --2- 172.21.15.12:0/2834578862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae10006720 0x7fae10008ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 --2- 172.21.15.12:0/2834578862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae10009380 0x7fae10009820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae280cd3e0 con 0x7fae10006720 2023-07-08T12:38:35.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1000a100 con 0x7fae10009380 2023-07-08T12:38:35.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.609+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1000a2c0 con 0x7fae10003af0 2023-07-08T12:38:35.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.610+0000 7fae2659c700 1 --2- 172.21.15.12:0/2834578862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae10006720 0x7fae10008ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.610+0000 7fae2659c700 1 -- 172.21.15.12:0/2834578862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae10003af0 msgr2=0x7fae10003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-08T12:38:35.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.610+0000 7fae2659c700 1 --2- 172.21.15.12:0/2834578862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae10003af0 0x7fae10003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.610+0000 7fae2659c700 1 -- 172.21.15.12:0/2834578862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae10009380 msgr2=0x7fae10009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.610+0000 7fae2659c700 1 --2- 172.21.15.12:0/2834578862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae10009380 0x7fae10009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.610+0000 7fae2659c700 1 -- 172.21.15.12:0/2834578862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae280cd6c0 con 0x7fae10006720 2023-07-08T12:38:35.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.611+0000 7fae2659c700 1 --2- 172.21.15.12:0/2834578862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae10006720 0x7fae10008ba0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae20007c80 tx=0x7fae10003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e24258335a58315 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.611+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae200072d0 con 0x7fae10006720 2023-07-08T12:38:35.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.611+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2000c710 con 0x7fae10006720 2023-07-08T12:38:35.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.611+0000 7fae25d9b700 1 -- 172.21.15.12:0/2834578862 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2000cac0 con 0x7fae10006720 2023-07-08T12:38:35.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.611+0000 7fae2759e700 1 -- 172.21.15.12:0/2834578862 reap_dead start 2023-07-08T12:38:35.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.612+0000 7fae2d21a700 1 -- 172.21.15.12:0/2834578862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae10006720 msgr2=0x7fae10008ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.612+0000 7fae2d21a700 1 --2- 172.21.15.12:0/2834578862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae10006720 0x7fae10008ba0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae20007c80 tx=0x7fae10003780 comp rx=0 tx=0).stop 2023-07-08T12:38:35.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.615+0000 7fae2d21a700 1 -- 172.21.15.12:0/2834578862 shutdown_connections 2023-07-08T12:38:35.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.615+0000 7fae2d21a700 1 --2- 172.21.15.12:0/2834578862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae10006720 0x7fae10008ba0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.615+0000 7fae2d21a700 1 -- 172.21.15.12:0/2834578862 >> 172.21.15.12:0/2834578862 conn(0x7fae2806abc0 msgr2=0x7fae280b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.615+0000 7fae2d21a700 1 -- 172.21.15.12:0/2834578862 shutdown_connections 2023-07-08T12:38:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.615+0000 7fae2d21a700 1 -- 172.21.15.12:0/2834578862 wait complete. 2023-07-08T12:38:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.617+0000 7fae2d21a700 1 Processor -- start 2023-07-08T12:38:35.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.617+0000 7fae2d21a700 1 -- start start 2023-07-08T12:38:35.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.618+0000 7fae2d21a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae10006720 0x7fae2815a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.618+0000 7fae2d21a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae28152820 0x7fae281ed340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.618+0000 7fae2d21a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae280c7af0 0x7fae280c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.618+0000 7fae2d21a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae280c84a0 con 0x7fae28152820 2023-07-08T12:38:35.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.618+0000 7fae2d21a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae280c6f50 con 0x7fae280c7af0 2023-07-08T12:38:35.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.618+0000 7fae2d21a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2815aae0 con 0x7fae10006720 2023-07-08T12:38:35.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.619+0000 7fae2759e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae280c7af0 0x7fae280c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.619+0000 7fae2759e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae280c7af0 0x7fae280c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48514/0 (socket says 172.21.15.12:48514) 2023-07-08T12:38:35.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.619+0000 7fae2759e700 1 -- 172.21.15.12:0/2010836567 learned_addr learned my addr 172.21.15.12:0/2010836567 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.619+0000 7fae2759e700 1 -- 172.21.15.12:0/2010836567 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae10006720 msgr2=0x7fae2815a5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-08T12:38:35.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.619+0000 7fae2759e700 1 --2- 172.21.15.12:0/2010836567 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae10006720 0x7fae2815a5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.619+0000 7fae2759e700 1 -- 172.21.15.12:0/2010836567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae28152820 msgr2=0x7fae281ed340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.619+0000 7fae2759e700 1 --2- 172.21.15.12:0/2010836567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae28152820 0x7fae281ed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.619+0000 7fae2759e700 1 -- 172.21.15.12:0/2010836567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae280c63d0 con 0x7fae280c7af0 2023-07-08T12:38:35.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.620+0000 7fae2759e700 1 --2- 172.21.15.12:0/2010836567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae280c7af0 0x7fae280c7f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae280ae2d0 tx=0x7fae1800ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.620+0000 7fae0f7fe700 1 -- 172.21.15.12:0/2010836567 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18005c60 con 0x7fae280c7af0 2023-07-08T12:38:35.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.620+0000 7fae0f7fe700 1 -- 172.21.15.12:0/2010836567 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae18005de0 con 0x7fae280c7af0 2023-07-08T12:38:35.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.620+0000 7fae0f7fe700 1 -- 172.21.15.12:0/2010836567 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae180072b0 con 0x7fae280c7af0 2023-07-08T12:38:35.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.621+0000 7fae2d21a700 1 -- 172.21.15.12:0/2010836567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae280c66b0 con 0x7fae280c7af0 2023-07-08T12:38:35.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.621+0000 7fae2d21a700 1 -- 172.21.15.12:0/2010836567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae280c6c70 con 0x7fae280c7af0 2023-07-08T12:38:35.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.625+0000 7fae0f7fe700 1 -- 172.21.15.12:0/2010836567 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fae18007410 con 0x7fae280c7af0 2023-07-08T12:38:35.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.622+0000 7f56b404f700 1 Processor -- start 2023-07-08T12:38:35.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.622+0000 7f56b404f700 1 -- start start 2023-07-08T12:38:35.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.624+0000 7f56b404f700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f56ac16c7e0 0x7f56ac0cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.624+0000 7f56b404f700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f56ac0cb8a0 0x7f56ac0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.624+0000 7f56b404f700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ac0ce620 con 0x7f56ac089570 2023-07-08T12:38:35.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.624+0000 7f56b404f700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ac089e70 con 0x7f56ac0cb8a0 2023-07-08T12:38:35.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.624+0000 7f56b404f700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ac0caac0 con 0x7f56ac16c7e0 2023-07-08T12:38:35.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.624+0000 7f56b25ec700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f56ac0cb8a0 0x7f56ac0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.624+0000 7f56b25ec700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f56ac0cb8a0 0x7f56ac0ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49778/0 (socket says 172.21.15.12:49778) 2023-07-08T12:38:35.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.624+0000 7f56b25ec700 1 -- 172.21.15.12:0/1402760522 learned_addr learned my addr 172.21.15.12:0/1402760522 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.625+0000 7f56b1deb700 1 --2- 172.21.15.12:0/1402760522 >> v2:172.21.15.160:3300/0 conn(0x7f56ac16c7e0 0x7f56ac0cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.626+0000 7fae2d21a700 1 -- 172.21.15.12:0/2010836567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae28157bc0 con 0x7fae280c7af0 2023-07-08T12:38:35.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.626+0000 7fae0f7fe700 1 --2- 172.21.15.12:0/2010836567 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fae1406ad30 0x7fae1406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.626+0000 7fae0f7fe700 1 -- 172.21.15.12:0/2010836567 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fae18088df0 con 0x7fae280c7af0 2023-07-08T12:38:35.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.626+0000 7f56b25ec700 1 -- 172.21.15.12:0/1402760522 >> v2:172.21.15.160:3300/0 conn(0x7f56ac16c7e0 msgr2=0x7f56ac0cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.626+0000 7f56b25ec700 1 --2- 172.21.15.12:0/1402760522 >> v2:172.21.15.160:3300/0 conn(0x7f56ac16c7e0 0x7f56ac0cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.626+0000 7f56b25ec700 1 -- 172.21.15.12:0/1402760522 >> v1:172.21.15.12:6789/0 conn(0x7f56ac089570 legacy=0x7f56ac0cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.626+0000 7f56b25ec700 1 -- 172.21.15.12:0/1402760522 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac0cc970 con 0x7f56ac0cb8a0 2023-07-08T12:38:35.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.626+0000 7f56b25ec700 1 --2- 172.21.15.12:0/1402760522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ac0cb8a0 0x7f56ac0ce0e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0b0dc0 tx=0x7f56a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c293190387c015 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.627+0000 7f56b0de9700 1 -- 172.21.15.12:0/1402760522 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a4003a20 con 0x7f56ac0cb8a0 2023-07-08T12:38:35.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.627+0000 7f56b0de9700 1 -- 172.21.15.12:0/1402760522 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a4003b80 con 0x7f56ac0cb8a0 2023-07-08T12:38:35.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.627+0000 7f56b0de9700 1 -- 172.21.15.12:0/1402760522 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a400bab0 con 0x7f56ac0cb8a0 2023-07-08T12:38:35.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.627+0000 7fae26d9d700 1 --2- 172.21.15.12:0/2010836567 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fae1406ad30 0x7fae1406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.628+0000 7f56b404f700 1 -- 172.21.15.12:0/1402760522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ac0cb8a0 msgr2=0x7f56ac0ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.628+0000 7f56b404f700 1 --2- 172.21.15.12:0/1402760522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ac0cb8a0 0x7f56ac0ce0e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0b0dc0 tx=0x7f56a400cce0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.628+0000 7fae26d9d700 1 --2- 172.21.15.12:0/2010836567 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fae1406ad30 0x7fae1406d1e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0096f0 tx=0x7fae1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.628+0000 7f56b404f700 1 -- 172.21.15.12:0/1402760522 shutdown_connections 2023-07-08T12:38:35.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.628+0000 7f56b404f700 1 --2- 172.21.15.12:0/1402760522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ac0cb8a0 0x7f56ac0ce0e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.628+0000 7f56b404f700 1 --2- 172.21.15.12:0/1402760522 >> v2:172.21.15.160:3300/0 conn(0x7f56ac16c7e0 0x7f56ac0cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.628+0000 7f56b404f700 1 -- 172.21.15.12:0/1402760522 >> 172.21.15.12:0/1402760522 conn(0x7f56ac06abc0 msgr2=0x7f56ac0b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.632+0000 7fae0f7fe700 1 -- 172.21.15.12:0/2010836567 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fae28157bc0 con 0x7fae280c7af0 2023-07-08T12:38:35.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.632+0000 7fb9a258a700 1 Processor -- start 2023-07-08T12:38:35.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.633+0000 7fb9a258a700 1 -- start start 2023-07-08T12:38:35.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.634+0000 7fb9a258a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c15a480 0x7fb99c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.634+0000 7fb9a258a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb99c0b2410 0x7fb99c0b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.634+0000 7fb9a258a700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb99c15def0 0x7fb99c15e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.634+0000 7fb9a258a700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99c15a850 con 0x7fb99c0b2410 2023-07-08T12:38:35.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.634+0000 7fb9a258a700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99c0b2da0 con 0x7fb99c15def0 2023-07-08T12:38:35.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.634+0000 7fb9a258a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0cde90 con 0x7fb99c15a480 2023-07-08T12:38:35.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.634+0000 7fb99b7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb99c0b2410 0x7fb99c0b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.634+0000 7fb99bfff700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c15a480 0x7fb99c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.634+0000 7fb99bfff700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c15a480 0x7fb99c0cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59400/0 (socket says 172.21.15.12:59400) 2023-07-08T12:38:35.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.634+0000 7fb99b7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb99c0b2410 0x7fb99c0b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49784/0 (socket says 172.21.15.12:49784) 2023-07-08T12:38:35.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.634+0000 7fb99bfff700 1 -- 172.21.15.12:0/2639664579 learned_addr learned my addr 172.21.15.12:0/2639664579 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.634+0000 7fb9a0b27700 1 --2- 172.21.15.12:0/2639664579 >> v2:172.21.15.160:3300/0 conn(0x7fb99c15def0 0x7fb99c15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.635+0000 7fb99bfff700 1 -- 172.21.15.12:0/2639664579 >> v2:172.21.15.160:3300/0 conn(0x7fb99c15def0 msgr2=0x7fb99c15e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.635+0000 7fb99bfff700 1 --2- 172.21.15.12:0/2639664579 >> v2:172.21.15.160:3300/0 conn(0x7fb99c15def0 0x7fb99c15e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.635+0000 7fb99bfff700 1 -- 172.21.15.12:0/2639664579 >> v2:172.21.15.12:3300/0 conn(0x7fb99c0b2410 msgr2=0x7fb99c0b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.635+0000 7fb99bfff700 1 --2- 172.21.15.12:0/2639664579 >> v2:172.21.15.12:3300/0 conn(0x7fb99c0b2410 0x7fb99c0b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.635+0000 7fb99bfff700 1 -- 172.21.15.12:0/2639664579 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c0cd8d0 con 0x7fb99c15a480 2023-07-08T12:38:35.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.635+0000 7fb99bfff700 1 --2- 172.21.15.12:0/2639664579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c15a480 0x7fb99c0cafe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0af8e0 tx=0x7fb98c009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c871c59ea9069969 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.635+0000 7fb99affd700 1 -- 172.21.15.12:0/2639664579 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c011830 con 0x7fb99c15a480 2023-07-08T12:38:35.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.635+0000 7fb99affd700 1 -- 172.21.15.12:0/2639664579 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c011990 con 0x7fb99c15a480 2023-07-08T12:38:35.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.635+0000 7fb99affd700 1 -- 172.21.15.12:0/2639664579 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c011d20 con 0x7fb99c15a480 2023-07-08T12:38:35.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.636+0000 7fb9a258a700 1 -- 172.21.15.12:0/2639664579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c15a480 msgr2=0x7fb99c0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.636+0000 7fb9a258a700 1 --2- 172.21.15.12:0/2639664579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c15a480 0x7fb99c0cafe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0af8e0 tx=0x7fb98c009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.636+0000 7f2003440700 1 Processor -- start 2023-07-08T12:38:35.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.636+0000 7fb9a258a700 1 -- 172.21.15.12:0/2639664579 shutdown_connections 2023-07-08T12:38:35.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.636+0000 7fb9a258a700 1 --2- 172.21.15.12:0/2639664579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c15a480 0x7fb99c0cafe0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.636+0000 7fb9a258a700 1 --2- 172.21.15.12:0/2639664579 >> v2:172.21.15.160:3300/0 conn(0x7fb99c15def0 0x7fb99c15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.636+0000 7fb9a258a700 1 --2- 172.21.15.12:0/2639664579 >> v2:172.21.15.12:3300/0 conn(0x7fb99c0b2410 0x7fb99c0b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.636+0000 7fb9a258a700 1 -- 172.21.15.12:0/2639664579 >> 172.21.15.12:0/2639664579 conn(0x7fb99c06ab60 msgr2=0x7fb99c06e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.637+0000 7fb9a258a700 1 -- 172.21.15.12:0/2639664579 shutdown_connections 2023-07-08T12:38:35.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.637+0000 7fb9a258a700 1 -- 172.21.15.12:0/2639664579 wait complete. 2023-07-08T12:38:35.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.636+0000 7f2003440700 1 -- start start 2023-07-08T12:38:35.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.637+0000 7ff8082be700 1 Processor -- start 2023-07-08T12:38:35.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.637+0000 7f56b404f700 1 -- 172.21.15.12:0/1402760522 shutdown_connections 2023-07-08T12:38:35.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.637+0000 7f56b404f700 1 -- 172.21.15.12:0/1402760522 wait complete. 2023-07-08T12:38:35.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.637+0000 7ff8082be700 1 -- start start 2023-07-08T12:38:35.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7ff8082be700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff800160640 0x7ff8000cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7ff8082be700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff800160a10 0x7ff8000c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7ff8082be700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8000cd990 con 0x7ff8000c3ce0 2023-07-08T12:38:35.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7ff8082be700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8000cde20 con 0x7ff800160a10 2023-07-08T12:38:35.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7ff8082be700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8000c40e0 con 0x7ff800160640 2023-07-08T12:38:35.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.638+0000 7f2003440700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ffc160700 0x7f1ffc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.638+0000 7f2003440700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1ffc0ccda0 0x7f1ffc06a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.638+0000 7f2003440700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc160ad0 con 0x7f1ffc0c27b0 2023-07-08T12:38:35.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.638+0000 7f2003440700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ffc06ff20 con 0x7f1ffc0ccda0 2023-07-08T12:38:35.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.638+0000 7f2003440700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc0cd560 con 0x7f1ffc160700 2023-07-08T12:38:35.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.638+0000 7f20011dc700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ffc160700 0x7f1ffc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.638+0000 7f20011dc700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ffc160700 0x7f1ffc069f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59406/0 (socket says 172.21.15.12:59406) 2023-07-08T12:38:35.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.638+0000 7f20011dc700 1 -- 172.21.15.12:0/924618748 learned_addr learned my addr 172.21.15.12:0/924618748 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.638+0000 7f20011dc700 1 -- 172.21.15.12:0/924618748 >> v2:172.21.15.160:3300/0 conn(0x7f1ffc0ccda0 msgr2=0x7f1ffc06a470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-08T12:38:35.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.638+0000 7f20011dc700 1 --2- 172.21.15.12:0/924618748 >> v2:172.21.15.160:3300/0 conn(0x7f1ffc0ccda0 0x7f1ffc06a470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.638+0000 7f20011dc700 1 -- 172.21.15.12:0/924618748 >> v1:172.21.15.160:6789/0 conn(0x7f1ffc0c27b0 legacy=0x7f1ffc0b1290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-08T12:38:35.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.638+0000 7f20011dc700 1 -- 172.21.15.12:0/924618748 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc164c50 con 0x7f1ffc160700 2023-07-08T12:38:35.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7f20011dc700 1 --2- 172.21.15.12:0/924618748 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ffc160700 0x7f1ffc069f30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80098f0 tx=0x7f1ff8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a69a9f194840dd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7f56b404f700 1 Processor -- start 2023-07-08T12:38:35.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7f56b404f700 1 -- start start 2023-07-08T12:38:35.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.638+0000 7fb9a258a700 1 Processor -- start 2023-07-08T12:38:35.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7fb9a258a700 1 -- start start 2023-07-08T12:38:35.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7ff80605a700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff800160640 0x7ff8000cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7ff80605a700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff800160640 0x7ff8000cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48536/0 (socket says 172.21.15.12:48536) 2023-07-08T12:38:35.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7ff80605a700 1 -- 172.21.15.12:0/3541157578 learned_addr learned my addr 172.21.15.12:0/3541157578 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7ff80605a700 1 -- 172.21.15.12:0/3541157578 >> v2:172.21.15.12:3300/0 conn(0x7ff800160a10 msgr2=0x7ff8000c37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7ff80605a700 1 --2- 172.21.15.12:0/3541157578 >> v2:172.21.15.12:3300/0 conn(0x7ff800160a10 0x7ff8000c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7ff80605a700 1 -- 172.21.15.12:0/3541157578 >> v1:172.21.15.12:6789/0 conn(0x7ff8000c3ce0 legacy=0x7ff8000b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7ff80605a700 1 -- 172.21.15.12:0/3541157578 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8000cf570 con 0x7ff800160640 2023-07-08T12:38:35.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7fb9a258a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb99c0b2410 0x7fb99c1f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7fb9a258a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c15a480 0x7fb99c15d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7fb9a258a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb99c15def0 0x7fb99c15d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7fb9a258a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c202f00 con 0x7fb99c0b2410 2023-07-08T12:38:35.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7fb9a258a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c15dc10 con 0x7fb99c15def0 2023-07-08T12:38:35.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7fb9a258a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0c50e0 con 0x7fb99c15a480 2023-07-08T12:38:35.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7fb99bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb99c0b2410 0x7fb99c1f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7fb99bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb99c0b2410 0x7fb99c1f5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49798/0 (socket says 172.21.15.12:49798) 2023-07-08T12:38:35.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7fb99bfff700 1 -- 172.21.15.12:0/2461141928 learned_addr learned my addr 172.21.15.12:0/2461141928 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7fb99b7fe700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c15a480 0x7fb99c15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7f56b404f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56ac089570 0x7f56ac06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7f56b404f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ac0cb8a0 0x7f56ac0c4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7f56b404f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56ac16c7e0 0x7f56ac0c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7f56b404f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac15c000 con 0x7f56ac0cb8a0 2023-07-08T12:38:35.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7fb9a0b27700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb99c15def0 0x7fb99c15d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7f56b404f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac15c2e0 con 0x7f56ac16c7e0 2023-07-08T12:38:35.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.640+0000 7f56b404f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac15c5c0 con 0x7f56ac089570 2023-07-08T12:38:35.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7f41876aa700 1 Processor -- start 2023-07-08T12:38:35.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.639+0000 7f41876aa700 1 -- start start 2023-07-08T12:38:35.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f41876aa700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f41800cb8c0 0x7f41800ce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f41876aa700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41800c1d30 con 0x7f4180160ab0 2023-07-08T12:38:35.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f41876aa700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41800cc840 con 0x7f41801606e0 2023-07-08T12:38:35.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f56b1deb700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56ac089570 0x7f56ac06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f56b1deb700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56ac089570 0x7f56ac06d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59422/0 (socket says 172.21.15.12:59422) 2023-07-08T12:38:35.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f56b1deb700 1 -- 172.21.15.12:0/1930686030 learned_addr learned my addr 172.21.15.12:0/1930686030 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f56b25ec700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56ac16c7e0 0x7f56ac0c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f41876aa700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41800ce670 con 0x7f41800cb8c0 2023-07-08T12:38:35.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7fb99b7fe700 1 -- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb99c15def0 msgr2=0x7fb99c15d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7fb99b7fe700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb99c15def0 0x7fb99c15d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7fb99b7fe700 1 -- 172.21.15.12:0/2461141928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb99c0b2410 msgr2=0x7fb99c1f5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7fb99b7fe700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb99c0b2410 0x7fb99c1f5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7fb99b7fe700 1 -- 172.21.15.12:0/2461141928 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c009550 con 0x7fb99c15a480 2023-07-08T12:38:35.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7ff80605a700 1 --2- 172.21.15.12:0/3541157578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff800160640 0x7ff8000cafe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff8000b34c0 tx=0x7ff7f800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=508799282a9316c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f56b15ea700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ac0cb8a0 0x7f56ac0c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f56b1deb700 1 -- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56ac16c7e0 msgr2=0x7f56ac0c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f56b1deb700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56ac16c7e0 0x7f56ac0c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f56b1deb700 1 -- 172.21.15.12:0/1930686030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ac0cb8a0 msgr2=0x7f56ac0c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f56b1deb700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ac0cb8a0 0x7f56ac0c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f56b1deb700 1 -- 172.21.15.12:0/1930686030 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a400c660 con 0x7f56ac089570 2023-07-08T12:38:35.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f56b15ea700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ac0cb8a0 0x7f56ac0c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:35.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7fb99b7fe700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c15a480 0x7fb99c15d190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb99000e870 tx=0x7fb99000eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f4184c45700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4180160ab0 0x7f41800cb1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37206/0 (socket says 172.21.15.12:37206) 2023-07-08T12:38:35.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7f4184c45700 1 -- 172.21.15.12:0/2410157950 learned_addr learned my addr 172.21.15.12:0/2410157950 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:35.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7fb998ff9700 1 -- 172.21.15.12:0/2461141928 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb990013680 con 0x7fb99c15a480 2023-07-08T12:38:35.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7fb998ff9700 1 -- 172.21.15.12:0/2461141928 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9900137e0 con 0x7fb99c15a480 2023-07-08T12:38:35.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7fb998ff9700 1 -- 172.21.15.12:0/2461141928 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb990013b50 con 0x7fb99c15a480 2023-07-08T12:38:35.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7fb9a258a700 1 -- 172.21.15.12:0/2461141928 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99c202280 con 0x7fb99c15a480 2023-07-08T12:38:35.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7fb9a258a700 1 -- 172.21.15.12:0/2461141928 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c202b20 con 0x7fb99c15a480 2023-07-08T12:38:35.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 -- 172.21.15.12:0/2410157950 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690692896 0 0) 0x7f41800cc840 con 0x7f41801606e0 2023-07-08T12:38:35.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 -- 172.21.15.12:0/2410157950 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f416c0035b0 con 0x7f41801606e0 2023-07-08T12:38:35.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 -- 172.21.15.12:0/2410157950 <== mon.1 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1600750598 0 0) 0x7f416c0035b0 con 0x7f41801606e0 2023-07-08T12:38:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 -- 172.21.15.12:0/2410157950 >> v2:172.21.15.12:3300/0 conn(0x7f41800cb8c0 msgr2=0x7f41800ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 --2- 172.21.15.12:0/2410157950 >> v2:172.21.15.12:3300/0 conn(0x7f41800cb8c0 0x7f41800ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 -- 172.21.15.12:0/2410157950 >> v1:172.21.15.12:6789/0 conn(0x7f4180160ab0 legacy=0x7f41800cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 -- 172.21.15.12:0/2410157950 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41800adc60 con 0x7f41801606e0 2023-07-08T12:38:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 -- 172.21.15.12:0/2410157950 <== mon.1 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f417c001670 con 0x7f41801606e0 2023-07-08T12:38:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 -- 172.21.15.12:0/2410157950 >> v1:172.21.15.160:6789/0 conn(0x7f41801606e0 legacy=0x7f418006c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 --2- 172.21.15.12:0/2410157950 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f416c003a00 0x7f416c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 --2- 172.21.15.12:0/2410157950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c004850 0x7f416c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 --2- 172.21.15.12:0/2410157950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f416c0052d0 0x7f416c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 -- 172.21.15.12:0/2410157950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c00c1d0 con 0x7f416c004850 2023-07-08T12:38:35.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 -- 172.21.15.12:0/2410157950 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c00c340 con 0x7f416c0052d0 2023-07-08T12:38:35.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7f4177fff700 1 -- 172.21.15.12:0/2410157950 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416c00c500 con 0x7f416c003a00 2023-07-08T12:38:35.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.643+0000 7f4185446700 1 --2- 172.21.15.12:0/2410157950 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f416c003a00 0x7f416c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.643+0000 7f4184c45700 1 --2- 172.21.15.12:0/2410157950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c004850 0x7f416c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7ff805058700 1 -- 172.21.15.12:0/3541157578 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800e070 con 0x7ff800160640 2023-07-08T12:38:35.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7ff805058700 1 -- 172.21.15.12:0/3541157578 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f800b9c0 con 0x7ff800160640 2023-07-08T12:38:35.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.641+0000 7ff805058700 1 -- 172.21.15.12:0/3541157578 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800bd70 con 0x7ff800160640 2023-07-08T12:38:35.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7ff8082be700 1 -- 172.21.15.12:0/3541157578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff800160640 msgr2=0x7ff8000cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.642+0000 7ff8082be700 1 --2- 172.21.15.12:0/3541157578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff800160640 0x7ff8000cafe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff8000b34c0 tx=0x7ff7f800b330 comp rx=0 tx=0).stop 2023-07-08T12:38:35.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.643+0000 7f1ff3fff700 1 -- 172.21.15.12:0/924618748 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8004230 con 0x7f1ffc160700 2023-07-08T12:38:35.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.643+0000 7f1ff3fff700 1 -- 172.21.15.12:0/924618748 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff8004390 con 0x7f1ffc160700 2023-07-08T12:38:35.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.643+0000 7f1ff3fff700 1 -- 172.21.15.12:0/924618748 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff80036a0 con 0x7f1ffc160700 2023-07-08T12:38:35.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f4185446700 1 -- 172.21.15.12:0/2410157950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f416c0052d0 msgr2=0x7f416c00ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-08T12:38:35.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f4185446700 1 --2- 172.21.15.12:0/2410157950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f416c0052d0 0x7f416c00ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f4185446700 1 -- 172.21.15.12:0/2410157950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c004850 msgr2=0x7f416c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f4185446700 1 --2- 172.21.15.12:0/2410157950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c004850 0x7f416c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f4185446700 1 -- 172.21.15.12:0/2410157950 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41800adc60 con 0x7f416c003a00 2023-07-08T12:38:35.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7fb9a258a700 1 -- 172.21.15.12:0/2461141928 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb99c203040 con 0x7fb99c15a480 2023-07-08T12:38:35.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f4185446700 1 --2- 172.21.15.12:0/2410157950 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f416c003a00 0x7f416c003e10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c1b0 tx=0x7f416c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8244b02ece081df server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7ff8082be700 1 -- 172.21.15.12:0/3541157578 shutdown_connections 2023-07-08T12:38:35.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7ff8082be700 1 --2- 172.21.15.12:0/3541157578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff800160640 0x7ff8000cafe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7ff8082be700 1 --2- 172.21.15.12:0/3541157578 >> v2:172.21.15.12:3300/0 conn(0x7ff800160a10 0x7ff8000c37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7ff8082be700 1 -- 172.21.15.12:0/3541157578 >> 172.21.15.12:0/3541157578 conn(0x7ff80006ab60 msgr2=0x7ff80006e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f2003440700 1 -- 172.21.15.12:0/924618748 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ffc160700 msgr2=0x7f1ffc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f2003440700 1 --2- 172.21.15.12:0/924618748 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ffc160700 0x7f1ffc069f30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80098f0 tx=0x7f1ff8009c00 comp rx=0 tx=0).stop 2023-07-08T12:38:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.643+0000 7f90210c7700 1 Processor -- start 2023-07-08T12:38:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.643+0000 7f90210c7700 1 -- start start 2023-07-08T12:38:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f2003440700 1 -- 172.21.15.12:0/924618748 shutdown_connections 2023-07-08T12:38:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f2003440700 1 --2- 172.21.15.12:0/924618748 >> v2:172.21.15.160:3300/0 conn(0x7f1ffc0ccda0 0x7f1ffc06a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f2003440700 1 --2- 172.21.15.12:0/924618748 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ffc160700 0x7f1ffc069f30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f2003440700 1 -- 172.21.15.12:0/924618748 >> 172.21.15.12:0/924618748 conn(0x7f1ffc0cc290 msgr2=0x7f1ffc0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f56b1deb700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56ac089570 0x7f56ac06d4d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006e70 tx=0x7f56a800dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f4177fff700 1 -- 172.21.15.12:0/2410157950 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c00cd10 con 0x7f416c003a00 2023-07-08T12:38:35.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f4177fff700 1 -- 172.21.15.12:0/2410157950 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c00ce90 con 0x7f416c003a00 2023-07-08T12:38:35.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7f4185c47700 1 -- 172.21.15.12:0/2410157950 reap_dead start 2023-07-08T12:38:35.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f56a27fc700 1 -- 172.21.15.12:0/1930686030 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8009b70 con 0x7f56ac089570 2023-07-08T12:38:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f56a27fc700 1 -- 172.21.15.12:0/1930686030 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a8009cd0 con 0x7f56ac089570 2023-07-08T12:38:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f56b404f700 1 -- 172.21.15.12:0/1930686030 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ac15c8a0 con 0x7f56ac089570 2023-07-08T12:38:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f56b404f700 1 -- 172.21.15.12:0/1930686030 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ac15bc60 con 0x7f56ac089570 2023-07-08T12:38:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.644+0000 7fb998ff9700 1 -- 172.21.15.12:0/2461141928 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb990013cb0 con 0x7fb99c15a480 2023-07-08T12:38:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7fb998ff9700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb98806adb0 0x7fb98806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7fb998ff9700 1 -- 172.21.15.12:0/2461141928 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb9900894a0 con 0x7fb99c15a480 2023-07-08T12:38:35.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7ff8082be700 1 -- 172.21.15.12:0/3541157578 shutdown_connections 2023-07-08T12:38:35.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f90210c7700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f901c0b1c70 0x7f901c0cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f90210c7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f901c0ce8c0 0x7f901c071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f90210c7700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901c15d230 con 0x7f901c15d6c0 2023-07-08T12:38:35.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f90210c7700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901c0c9e60 con 0x7f901c0b1c70 2023-07-08T12:38:35.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f90210c7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0cd920 con 0x7f901c0ce8c0 2023-07-08T12:38:35.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7ff8082be700 1 -- 172.21.15.12:0/3541157578 wait complete. 2023-07-08T12:38:35.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f901b59e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f901c0ce8c0 0x7f901c071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7fb99bfff700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb98806adb0 0x7fb98806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f41876aa700 1 -- 172.21.15.12:0/2410157950 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f416c003a00 msgr2=0x7f416c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f41876aa700 1 --2- 172.21.15.12:0/2410157950 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f416c003a00 0x7f416c003e10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c1b0 tx=0x7f416c005880 comp rx=0 tx=0).stop 2023-07-08T12:38:35.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f41876aa700 1 -- 172.21.15.12:0/2410157950 shutdown_connections 2023-07-08T12:38:35.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f41876aa700 1 --2- 172.21.15.12:0/2410157950 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f416c003a00 0x7f416c003e10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f41876aa700 1 -- 172.21.15.12:0/2410157950 >> 172.21.15.12:0/2410157950 conn(0x7f418006abc0 msgr2=0x7f41800b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f901b59e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f901c0ce8c0 0x7f901c071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59442/0 (socket says 172.21.15.12:59442) 2023-07-08T12:38:35.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f901b59e700 1 -- 172.21.15.12:0/1235683519 learned_addr learned my addr 172.21.15.12:0/1235683519 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.645+0000 7f56a27fc700 1 -- 172.21.15.12:0/1930686030 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8019650 con 0x7f56ac089570 2023-07-08T12:38:35.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.646+0000 7f901b59e700 1 -- 172.21.15.12:0/1235683519 >> v2:172.21.15.160:3300/0 conn(0x7f901c0b1c70 msgr2=0x7f901c0cd3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-08T12:38:35.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.646+0000 7f901b59e700 1 --2- 172.21.15.12:0/1235683519 >> v2:172.21.15.160:3300/0 conn(0x7f901c0b1c70 0x7f901c0cd3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.646+0000 7f901b59e700 1 -- 172.21.15.12:0/1235683519 >> v1:172.21.15.160:6789/0 conn(0x7f901c15d6c0 legacy=0x7f901c0ce1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-08T12:38:35.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.646+0000 7f901b59e700 1 -- 172.21.15.12:0/1235683519 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c0cdbc0 con 0x7f901c0ce8c0 2023-07-08T12:38:35.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.646+0000 7f2003440700 1 -- 172.21.15.12:0/924618748 shutdown_connections 2023-07-08T12:38:35.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.646+0000 7f2003440700 1 -- 172.21.15.12:0/924618748 wait complete. 2023-07-08T12:38:35.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.646+0000 7f901b59e700 1 --2- 172.21.15.12:0/1235683519 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f901c0ce8c0 0x7f901c071530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9014009e50 tx=0x7f901400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5483a143c993c6f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.646+0000 7f41876aa700 1 -- 172.21.15.12:0/2410157950 shutdown_connections 2023-07-08T12:38:35.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.646+0000 7fb99bfff700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb98806adb0 0x7fb98806d260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009fd0 tx=0x7fb98c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.646+0000 7f41876aa700 1 -- 172.21.15.12:0/2410157950 wait complete. 2023-07-08T12:38:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.647+0000 7ff8082be700 1 Processor -- start 2023-07-08T12:38:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.647+0000 7ff8082be700 1 -- start start 2023-07-08T12:38:35.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7f41876aa700 1 Processor -- start 2023-07-08T12:38:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7f41876aa700 1 -- start start 2023-07-08T12:38:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7f2003440700 1 Processor -- start 2023-07-08T12:38:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7f2003440700 1 -- start start 2023-07-08T12:38:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7ff8082be700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8000c3ce0 0x7ff80014d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7ff8082be700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800160640 0x7ff80006be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7ff8082be700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff800160a10 0x7ff80014f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7ff8082be700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000b51c0 con 0x7ff800160640 2023-07-08T12:38:35.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7ff8082be700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000b6020 con 0x7ff8000c3ce0 2023-07-08T12:38:35.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7ff8082be700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000b5410 con 0x7ff800160a10 2023-07-08T12:38:35.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7ff80605a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8000c3ce0 0x7ff80014d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7ff80605a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8000c3ce0 0x7ff80014d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48572/0 (socket says 172.21.15.12:48572) 2023-07-08T12:38:35.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7ff80605a700 1 -- 172.21.15.12:0/3485861883 learned_addr learned my addr 172.21.15.12:0/3485861883 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.648+0000 7ff805859700 1 --2- 172.21.15.12:0/3485861883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800160640 0x7ff80006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7ff80605a700 1 -- 172.21.15.12:0/3485861883 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff800160a10 msgr2=0x7ff80014f170 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-08T12:38:35.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7ff80605a700 1 --2- 172.21.15.12:0/3485861883 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff800160a10 0x7ff80014f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7ff80605a700 1 -- 172.21.15.12:0/3485861883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800160640 msgr2=0x7ff80006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7ff80605a700 1 --2- 172.21.15.12:0/3485861883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800160640 0x7ff80006be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7ff80605a700 1 -- 172.21.15.12:0/3485861883 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f8009b30 con 0x7ff8000c3ce0 2023-07-08T12:38:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f41876aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c003a00 0x7f4180158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f41876aa700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f418006d550 0x7f4180155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f41876aa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f418006cbb0 0x7f41800af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f41876aa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41801563a0 con 0x7f416c003a00 2023-07-08T12:38:35.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f41876aa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418006dcc0 con 0x7f418006cbb0 2023-07-08T12:38:35.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f41876aa700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418006dfa0 con 0x7f418006d550 2023-07-08T12:38:35.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f4185446700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c003a00 0x7f4180158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f4185446700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c003a00 0x7f4180158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49834/0 (socket says 172.21.15.12:49834) 2023-07-08T12:38:35.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f4185446700 1 -- 172.21.15.12:0/41030716 learned_addr learned my addr 172.21.15.12:0/41030716 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f4184c45700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f418006d550 0x7f4180155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f4185c47700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f418006cbb0 0x7f41800af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7ff80605a700 1 --2- 172.21.15.12:0/3485861883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8000c3ce0 0x7ff80014d280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff8000c4220 tx=0x7ff7f8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.651+0000 7fb998ff9700 1 -- 172.21.15.12:0/2461141928 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb99c203040 con 0x7fb99c15a480 2023-07-08T12:38:35.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f4185446700 1 -- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f418006d550 msgr2=0x7f4180155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f4185446700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f418006d550 0x7f4180155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f4185446700 1 -- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f418006cbb0 msgr2=0x7f41800af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f4185446700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f418006cbb0 0x7f41800af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f4185446700 1 -- 172.21.15.12:0/41030716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418006e280 con 0x7f416c003a00 2023-07-08T12:38:35.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f4185446700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c003a00 0x7f4180158340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f417c014040 tx=0x7f417c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f4175ffb700 1 -- 172.21.15.12:0/41030716 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c001c20 con 0x7f416c003a00 2023-07-08T12:38:35.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f4175ffb700 1 -- 172.21.15.12:0/41030716 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c020070 con 0x7f416c003a00 2023-07-08T12:38:35.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f4175ffb700 1 -- 172.21.15.12:0/41030716 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c005520 con 0x7f416c003a00 2023-07-08T12:38:35.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.651+0000 7f41876aa700 1 -- 172.21.15.12:0/41030716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418006c3b0 con 0x7f416c003a00 2023-07-08T12:38:35.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.651+0000 7f41876aa700 1 -- 172.21.15.12:0/41030716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418006d9c0 con 0x7f416c003a00 2023-07-08T12:38:35.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.652+0000 7f41876aa700 1 -- 172.21.15.12:0/41030716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41600052f0 con 0x7f416c003a00 2023-07-08T12:38:35.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7ff7f6ffd700 1 -- 172.21.15.12:0/3485861883 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8004250 con 0x7ff8000c3ce0 2023-07-08T12:38:35.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7ff7f6ffd700 1 -- 172.21.15.12:0/3485861883 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f80043b0 con 0x7ff8000c3ce0 2023-07-08T12:38:35.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7ff7f6ffd700 1 -- 172.21.15.12:0/3485861883 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8004740 con 0x7ff8000c3ce0 2023-07-08T12:38:35.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7ff8082be700 1 -- 172.21.15.12:0/3485861883 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8000b56f0 con 0x7ff8000c3ce0 2023-07-08T12:38:35.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7ff8082be700 1 -- 172.21.15.12:0/3485861883 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8000b5cb0 con 0x7ff8000c3ce0 2023-07-08T12:38:35.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.651+0000 7ff8082be700 1 -- 172.21.15.12:0/3485861883 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8000b5e70 con 0x7ff8000c3ce0 2023-07-08T12:38:35.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.653+0000 7ff7f6ffd700 1 -- 172.21.15.12:0/3485861883 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff7f801c440 con 0x7ff8000c3ce0 2023-07-08T12:38:35.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.654+0000 7ff7f6ffd700 1 --2- 172.21.15.12:0/3485861883 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff7e406ad30 0x7ff7e406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.654+0000 7ff7f6ffd700 1 -- 172.21.15.12:0/3485861883 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff7f808b2e0 con 0x7ff8000c3ce0 2023-07-08T12:38:35.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.654+0000 7ff805859700 1 --2- 172.21.15.12:0/3485861883 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff7e406ad30 0x7ff7e406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f2003440700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ffc0c27b0 0x7f1ffc162f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f2003440700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ffc0ccda0 0x7f1ffc161cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f2003440700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ffc160700 0x7f1ffc14ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f2003440700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc162a90 con 0x7f1ffc160700 2023-07-08T12:38:35.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f2003440700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc15fee0 con 0x7f1ffc0c27b0 2023-07-08T12:38:35.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f2003440700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc162760 con 0x7f1ffc0ccda0 2023-07-08T12:38:35.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f20011dc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ffc0c27b0 0x7f1ffc162f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f20011dc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ffc0c27b0 0x7f1ffc162f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48584/0 (socket says 172.21.15.12:48584) 2023-07-08T12:38:35.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.649+0000 7f20011dc700 1 -- 172.21.15.12:0/2554776118 learned_addr learned my addr 172.21.15.12:0/2554776118 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f20019dd700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ffc160700 0x7f1ffc14ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f20009db700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ffc0ccda0 0x7f1ffc161cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f20011dc700 1 -- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ffc0ccda0 msgr2=0x7f1ffc161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f20011dc700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ffc0ccda0 0x7f1ffc161cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f20011dc700 1 -- 172.21.15.12:0/2554776118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ffc160700 msgr2=0x7f1ffc14ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f20011dc700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ffc160700 0x7f1ffc14ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f20011dc700 1 -- 172.21.15.12:0/2554776118 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff8009550 con 0x7f1ffc0c27b0 2023-07-08T12:38:35.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f20011dc700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ffc0c27b0 0x7f1ffc162f20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009c40 tx=0x7f1ff8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f1ff1ffb700 1 -- 172.21.15.12:0/2554776118 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800b660 con 0x7f1ffc0c27b0 2023-07-08T12:38:35.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f2003440700 1 -- 172.21.15.12:0/2554776118 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ffc0c68e0 con 0x7f1ffc0c27b0 2023-07-08T12:38:35.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.651+0000 7f2003440700 1 -- 172.21.15.12:0/2554776118 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc0c6ea0 con 0x7f1ffc0c27b0 2023-07-08T12:38:35.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f1ff1ffb700 1 -- 172.21.15.12:0/2554776118 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff800b7e0 con 0x7f1ffc0c27b0 2023-07-08T12:38:35.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.652+0000 7f1ff1ffb700 1 -- 172.21.15.12:0/2554776118 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800bb90 con 0x7f1ffc0c27b0 2023-07-08T12:38:35.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f9019d9b700 1 -- 172.21.15.12:0/1235683519 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901400e070 con 0x7f901c0ce8c0 2023-07-08T12:38:35.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f9019d9b700 1 -- 172.21.15.12:0/1235683519 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9014003a20 con 0x7f901c0ce8c0 2023-07-08T12:38:35.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.650+0000 7f9019d9b700 1 -- 172.21.15.12:0/1235683519 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014003dd0 con 0x7f901c0ce8c0 2023-07-08T12:38:35.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.652+0000 7f90210c7700 1 -- 172.21.15.12:0/1235683519 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f901c0ce8c0 msgr2=0x7f901c071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.652+0000 7f90210c7700 1 --2- 172.21.15.12:0/1235683519 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f901c0ce8c0 0x7f901c071530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9014009e50 tx=0x7f901400b330 comp rx=0 tx=0).stop 2023-07-08T12:38:35.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.652+0000 7f90210c7700 1 -- 172.21.15.12:0/1235683519 shutdown_connections 2023-07-08T12:38:35.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.652+0000 7f90210c7700 1 --2- 172.21.15.12:0/1235683519 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f901c0ce8c0 0x7f901c071530 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.652+0000 7f90210c7700 1 --2- 172.21.15.12:0/1235683519 >> v2:172.21.15.160:3300/0 conn(0x7f901c0b1c70 0x7f901c0cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.652+0000 7f90210c7700 1 -- 172.21.15.12:0/1235683519 >> 172.21.15.12:0/1235683519 conn(0x7f901c0cc290 msgr2=0x7f901c0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.655+0000 7ff805859700 1 --2- 172.21.15.12:0/3485861883 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff7e406ad30 0x7ff7e406d1e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff80014dcf0 tx=0x7ff7fc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.655+0000 7f56b404f700 1 -- 172.21.15.12:0/1930686030 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56940052f0 con 0x7f56ac089570 2023-07-08T12:38:35.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.655+0000 7f4175ffb700 1 -- 172.21.15.12:0/41030716 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f417c022030 con 0x7f416c003a00 2023-07-08T12:38:35.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.655+0000 7f4175ffb700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f416406ad30 0x7f416406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.656+0000 7f4175ffb700 1 -- 172.21.15.12:0/41030716 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f417c08abd0 con 0x7f416c003a00 2023-07-08T12:38:35.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.656+0000 7f4184c45700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f416406ad30 0x7f416406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.656+0000 7f4184c45700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f416406ad30 0x7f416406d1e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f41800c8990 tx=0x7f4170007e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.657+0000 7ff7f6ffd700 1 -- 172.21.15.12:0/3485861883 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff7f8056330 con 0x7ff8000c3ce0 2023-07-08T12:38:35.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.658+0000 7f56a27fc700 1 -- 172.21.15.12:0/1930686030 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f56a80197b0 con 0x7f56ac089570 2023-07-08T12:38:35.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.658+0000 7f56a27fc700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f569006ace0 0x7f569006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.659+0000 7f56a27fc700 1 -- 172.21.15.12:0/1930686030 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f56a8089bb0 con 0x7f56ac089570 2023-07-08T12:38:35.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.659+0000 7f56b15ea700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f569006ace0 0x7f569006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.658+0000 7f1ff1ffb700 1 -- 172.21.15.12:0/2554776118 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1ff8022070 con 0x7f1ffc0c27b0 2023-07-08T12:38:35.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.659+0000 7f1ff1ffb700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1fe006ad30 0x7f1fe006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.659+0000 7f1ff1ffb700 1 -- 172.21.15.12:0/2554776118 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1ff808b3f0 con 0x7f1ffc0c27b0 2023-07-08T12:38:35.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.659+0000 7f2003440700 1 -- 172.21.15.12:0/2554776118 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fdc0052f0 con 0x7f1ffc0c27b0 2023-07-08T12:38:35.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.659+0000 7f20009db700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1fe006ad30 0x7f1fe006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.660+0000 7f90210c7700 1 -- 172.21.15.12:0/1235683519 shutdown_connections 2023-07-08T12:38:35.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.660+0000 7f56b15ea700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f569006ace0 0x7f569006d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0b48c0 tx=0x7f569c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.660+0000 7f20009db700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1fe006ad30 0x7f1fe006d1e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0b2760 tx=0x7f1fec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.661+0000 7f90210c7700 1 -- 172.21.15.12:0/1235683519 wait complete. 2023-07-08T12:38:35.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.661+0000 7f4175ffb700 1 -- 172.21.15.12:0/41030716 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f417c055b00 con 0x7f416c003a00 2023-07-08T12:38:35.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.663+0000 7f90210c7700 1 Processor -- start 2023-07-08T12:38:35.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.663+0000 7f90210c7700 1 -- start start 2023-07-08T12:38:35.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.664+0000 7f56a27fc700 1 -- 172.21.15.12:0/1930686030 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f56a8054960 con 0x7f56ac089570 2023-07-08T12:38:35.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.664+0000 7f90210c7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f901c0b1c70 0x7f901c0c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.664+0000 7f90210c7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c0ce8c0 0x7f901c0c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.664+0000 7f90210c7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f901c0c94f0 0x7f901c1f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.664+0000 7f90210c7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0b19d0 con 0x7f901c0ce8c0 2023-07-08T12:38:35.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.664+0000 7f90210c7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0c5480 con 0x7f901c0b1c70 2023-07-08T12:38:35.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.664+0000 7f90210c7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0c5910 con 0x7f901c0c94f0 2023-07-08T12:38:35.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.664+0000 7f901ad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f901c0b1c70 0x7f901c0c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.664+0000 7f901ad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f901c0b1c70 0x7f901c0c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48598/0 (socket says 172.21.15.12:48598) 2023-07-08T12:38:35.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.664+0000 7f901ad9d700 1 -- 172.21.15.12:0/2562869102 learned_addr learned my addr 172.21.15.12:0/2562869102 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.665+0000 7f901ad9d700 1 -- 172.21.15.12:0/2562869102 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f901c0c94f0 msgr2=0x7f901c1f9b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-08T12:38:35.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.665+0000 7f901ad9d700 1 --2- 172.21.15.12:0/2562869102 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f901c0c94f0 0x7f901c1f9b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.665+0000 7f901ad9d700 1 -- 172.21.15.12:0/2562869102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c0ce8c0 msgr2=0x7f901c0c3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.665+0000 7f901ad9d700 1 --2- 172.21.15.12:0/2562869102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c0ce8c0 0x7f901c0c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.665+0000 7f901ad9d700 1 -- 172.21.15.12:0/2562869102 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9014009b30 con 0x7f901c0b1c70 2023-07-08T12:38:35.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.665+0000 7f901ad9d700 1 --2- 172.21.15.12:0/2562869102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f901c0b1c70 0x7f901c0c35b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f901000b550 tx=0x7f901000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.665+0000 7f90037fe700 1 -- 172.21.15.12:0/2562869102 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010011830 con 0x7f901c0b1c70 2023-07-08T12:38:35.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.665+0000 7f90037fe700 1 -- 172.21.15.12:0/2562869102 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90100119b0 con 0x7f901c0b1c70 2023-07-08T12:38:35.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.665+0000 7f90210c7700 1 -- 172.21.15.12:0/2562869102 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c0b1130 con 0x7f901c0b1c70 2023-07-08T12:38:35.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.665+0000 7f90037fe700 1 -- 172.21.15.12:0/2562869102 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010011d60 con 0x7f901c0b1c70 2023-07-08T12:38:35.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.665+0000 7f90210c7700 1 -- 172.21.15.12:0/2562869102 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c0b16f0 con 0x7f901c0b1c70 2023-07-08T12:38:35.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.667+0000 7f90210c7700 1 -- 172.21.15.12:0/2562869102 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c0c5760 con 0x7f901c0b1c70 2023-07-08T12:38:35.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.667+0000 7f90037fe700 1 -- 172.21.15.12:0/2562869102 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f901001a400 con 0x7f901c0b1c70 2023-07-08T12:38:35.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.668+0000 7f1ff1ffb700 1 -- 172.21.15.12:0/2554776118 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1ff8056260 con 0x7f1ffc0c27b0 2023-07-08T12:38:35.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.668+0000 7f90037fe700 1 --2- 172.21.15.12:0/2562869102 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f900806ace0 0x7f900806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.668+0000 7f90037fe700 1 -- 172.21.15.12:0/2562869102 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f90100898b0 con 0x7f901c0b1c70 2023-07-08T12:38:35.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.668+0000 7f901a59c700 1 --2- 172.21.15.12:0/2562869102 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f900806ace0 0x7f900806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.669+0000 7f901a59c700 1 --2- 172.21.15.12:0/2562869102 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f900806ace0 0x7f900806d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f901c16a340 tx=0x7f900c006c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.672+0000 7f90037fe700 1 -- 172.21.15.12:0/2562869102 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f90100545d0 con 0x7f901c0b1c70 2023-07-08T12:38:35.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.673+0000 7f4c2672f700 1 Processor -- start 2023-07-08T12:38:35.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.673+0000 7f4c2672f700 1 -- start start 2023-07-08T12:38:35.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.675+0000 7f4c2672f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c200c9970 0x7f4c200cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.675+0000 7f4c2672f700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4c200c9d40 0x7f4c200cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.675+0000 7f4c2672f700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c200cce20 con 0x7f4c200cb6e0 2023-07-08T12:38:35.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.675+0000 7f4c2672f700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c200cd100 con 0x7f4c200c9d40 2023-07-08T12:38:35.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.675+0000 7f4c2672f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200cd3e0 con 0x7f4c200c9970 2023-07-08T12:38:35.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.675+0000 7f4c24ccc700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4c200cb6e0 0x7f4c20157d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37258/0 (socket says 172.21.15.12:37258) 2023-07-08T12:38:35.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.675+0000 7f4c24ccc700 1 -- 172.21.15.12:0/1542934059 learned_addr learned my addr 172.21.15.12:0/1542934059 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:35.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.676+0000 7f4c1ffff700 1 --2- 172.21.15.12:0/1542934059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c200c9970 0x7f4c200cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.676+0000 7f4c1f7fe700 1 --2- 172.21.15.12:0/1542934059 >> v2:172.21.15.12:3300/0 conn(0x7f4c200c9d40 0x7f4c200cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.677+0000 7f4c1f7fe700 1 -- 172.21.15.12:0/1542934059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c200c9970 msgr2=0x7f4c200cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:35.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.677+0000 7f4c1f7fe700 1 --2- 172.21.15.12:0/1542934059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c200c9970 0x7f4c200cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.677+0000 7f4c1f7fe700 1 -- 172.21.15.12:0/1542934059 >> v1:172.21.15.12:6789/0 conn(0x7f4c200cb6e0 legacy=0x7f4c20157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.677+0000 7f4c1f7fe700 1 -- 172.21.15.12:0/1542934059 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c200cd6c0 con 0x7f4c200c9d40 2023-07-08T12:38:35.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.677+0000 7f4c1f7fe700 1 --2- 172.21.15.12:0/1542934059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c200c9d40 0x7f4c200cb1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009840 tx=0x7f4c14009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17dcf3177643a56c server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.678+0000 7f4c1effd700 1 -- 172.21.15.12:0/1542934059 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14011830 con 0x7f4c200c9d40 2023-07-08T12:38:35.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.678+0000 7f4c1effd700 1 -- 172.21.15.12:0/1542934059 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c14011990 con 0x7f4c200c9d40 2023-07-08T12:38:35.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.678+0000 7f4c1effd700 1 -- 172.21.15.12:0/1542934059 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14011d00 con 0x7f4c200c9d40 2023-07-08T12:38:35.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.679+0000 7f4c2672f700 1 -- 172.21.15.12:0/1542934059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c200c9d40 msgr2=0x7f4c200cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.679+0000 7f4c2672f700 1 --2- 172.21.15.12:0/1542934059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c200c9d40 0x7f4c200cb1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009840 tx=0x7f4c14009b50 comp rx=0 tx=0).stop 2023-07-08T12:38:35.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.681+0000 7f4c2672f700 1 -- 172.21.15.12:0/1542934059 shutdown_connections 2023-07-08T12:38:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.681+0000 7f4c2672f700 1 --2- 172.21.15.12:0/1542934059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c200c9970 0x7f4c200cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.681+0000 7f4c2672f700 1 --2- 172.21.15.12:0/1542934059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c200c9d40 0x7f4c200cb1a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.681+0000 7f4c2672f700 1 -- 172.21.15.12:0/1542934059 >> 172.21.15.12:0/1542934059 conn(0x7f4c2006abc0 msgr2=0x7f4c200b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.682+0000 7f4c2672f700 1 -- 172.21.15.12:0/1542934059 shutdown_connections 2023-07-08T12:38:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.682+0000 7f4c2672f700 1 -- 172.21.15.12:0/1542934059 wait complete. 2023-07-08T12:38:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.684+0000 7f4c2672f700 1 Processor -- start 2023-07-08T12:38:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.684+0000 7f4c2672f700 1 -- start start 2023-07-08T12:38:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.686+0000 7f4c2672f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c200c9970 0x7f4c2014eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.686+0000 7f4c2672f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c200c9d40 0x7f4c200c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.686+0000 7f4c2672f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c200cb6e0 0x7f4c200aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.686+0000 7f4c2672f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2014f530 con 0x7f4c200c9970 2023-07-08T12:38:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.686+0000 7f4c2672f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200b1690 con 0x7f4c200c9d40 2023-07-08T12:38:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.686+0000 7f4c2672f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200c6bc0 con 0x7f4c200cb6e0 2023-07-08T12:38:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.686+0000 7f4c1ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c200c9970 0x7f4c2014eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.686+0000 7f4c1ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c200c9970 0x7f4c2014eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49870/0 (socket says 172.21.15.12:49870) 2023-07-08T12:38:35.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.686+0000 7f4c1ffff700 1 -- 172.21.15.12:0/2192108546 learned_addr learned my addr 172.21.15.12:0/2192108546 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:35.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.687+0000 7f4c1f7fe700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c200c9d40 0x7f4c200c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.687+0000 7f4c24ccc700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c200cb6e0 0x7f4c200aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.687+0000 7f4c1ffff700 1 -- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c200cb6e0 msgr2=0x7f4c200aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.687+0000 7f4c1ffff700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c200cb6e0 0x7f4c200aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.687+0000 7f4c1ffff700 1 -- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c200c9d40 msgr2=0x7f4c200c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:35.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.687+0000 7f4c1ffff700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c200c9d40 0x7f4c200c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:35.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.687+0000 7f4c1ffff700 1 -- 172.21.15.12:0/2192108546 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c140094a0 con 0x7f4c200c9970 2023-07-08T12:38:35.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.687+0000 7f4c1ffff700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c200c9970 0x7f4c2014eff0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000d900 tx=0x7f4c1000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.692+0000 7f4c1cff9700 1 -- 172.21.15.12:0/2192108546 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1000b8d0 con 0x7f4c200c9970 2023-07-08T12:38:35.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.692+0000 7f4c1cff9700 1 -- 172.21.15.12:0/2192108546 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1000ba30 con 0x7f4c200c9970 2023-07-08T12:38:35.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.692+0000 7f4c1cff9700 1 -- 172.21.15.12:0/2192108546 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1000bdc0 con 0x7f4c200c9970 2023-07-08T12:38:35.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.692+0000 7f4c2672f700 1 -- 172.21.15.12:0/2192108546 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c200c6ea0 con 0x7f4c200c9970 2023-07-08T12:38:35.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.692+0000 7f4c2672f700 1 -- 172.21.15.12:0/2192108546 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c200c3b20 con 0x7f4c200c9970 2023-07-08T12:38:35.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.696+0000 7f4c1cff9700 1 -- 172.21.15.12:0/2192108546 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c10011460 con 0x7f4c200c9970 2023-07-08T12:38:35.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.696+0000 7f4c1cff9700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4c0406ad80 0x7f4c0406d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.696+0000 7f4c1cff9700 1 -- 172.21.15.12:0/2192108546 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4c100897f0 con 0x7f4c200c9970 2023-07-08T12:38:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.698+0000 7f4c1f7fe700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4c0406ad80 0x7f4c0406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.699+0000 7f4c2672f700 1 -- 172.21.15.12:0/2192108546 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc0052f0 con 0x7f4c200c9970 2023-07-08T12:38:35.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.708+0000 7f4c1cff9700 1 -- 172.21.15.12:0/2192108546 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4c10054620 con 0x7f4c200c9970 2023-07-08T12:38:35.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:35.731+0000 7f4c1f7fe700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4c0406ad80 0x7f4c0406d230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009b90 tx=0x7f4c14008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:35.866 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:38:35.902 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:38:36.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.055+0000 7fae2d21a700 1 -- 172.21.15.12:0/2010836567 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fae28157bc0 con 0x7fae280c7af0 2023-07-08T12:38:36.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.057+0000 7fae0f7fe700 1 -- 172.21.15.12:0/2010836567 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae28157bc0 con 0x7fae280c7af0 2023-07-08T12:38:36.060 INFO:teuthology.orchestra.run.smithi012.stdout:55834574858 2023-07-08T12:38:36.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.065+0000 7fb9a258a700 1 -- 172.21.15.12:0/2461141928 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb99c203040 con 0x7fb99c15a480 2023-07-08T12:38:36.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.065+0000 7fb998ff9700 1 -- 172.21.15.12:0/2461141928 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb99c203040 con 0x7fb99c15a480 2023-07-08T12:38:36.070 INFO:teuthology.orchestra.run.smithi012.stdout:55834574857 2023-07-08T12:38:36.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.070+0000 7fb9a258a700 1 -- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb98806adb0 msgr2=0x7fb98806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.070+0000 7fb9a258a700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb98806adb0 0x7fb98806d260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009fd0 tx=0x7fb98c008030 comp rx=0 tx=0).stop 2023-07-08T12:38:36.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.070+0000 7fb9a258a700 1 -- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c15a480 msgr2=0x7fb99c15d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.070+0000 7fb9a258a700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c15a480 0x7fb99c15d190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb99000e870 tx=0x7fb99000eb80 comp rx=0 tx=0).stop 2023-07-08T12:38:36.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.071+0000 7fae2d21a700 1 -- 172.21.15.12:0/2010836567 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fae1406ad30 msgr2=0x7fae1406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.071+0000 7fae2d21a700 1 --2- 172.21.15.12:0/2010836567 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fae1406ad30 0x7fae1406d1e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0096f0 tx=0x7fae1c008040 comp rx=0 tx=0).stop 2023-07-08T12:38:36.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.071+0000 7fae2d21a700 1 -- 172.21.15.12:0/2010836567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae280c7af0 msgr2=0x7fae280c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.071+0000 7fae2d21a700 1 --2- 172.21.15.12:0/2010836567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae280c7af0 0x7fae280c7f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae280ae2d0 tx=0x7fae1800ca90 comp rx=0 tx=0).stop 2023-07-08T12:38:36.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.075+0000 7fb9a258a700 1 -- 172.21.15.12:0/2461141928 shutdown_connections 2023-07-08T12:38:36.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.075+0000 7fb9a258a700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb98806adb0 0x7fb98806d260 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.075+0000 7fb9a258a700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb99c15def0 0x7fb99c15d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.075+0000 7fb9a258a700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb99c0b2410 0x7fb99c1f5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.075+0000 7fb9a258a700 1 --2- 172.21.15.12:0/2461141928 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c15a480 0x7fb99c15d190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.075+0000 7fb9a258a700 1 -- 172.21.15.12:0/2461141928 >> 172.21.15.12:0/2461141928 conn(0x7fb99c06ab60 msgr2=0x7fb99c0c4020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:36.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.075+0000 7ff8082be700 1 -- 172.21.15.12:0/3485861883 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff80004ea60 con 0x7ff8000c3ce0 2023-07-08T12:38:36.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.075+0000 7f41876aa700 1 -- 172.21.15.12:0/41030716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4160005870 con 0x7f416c003a00 2023-07-08T12:38:36.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.075+0000 7f4175ffb700 1 -- 172.21.15.12:0/41030716 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f417c059120 con 0x7f416c003a00 2023-07-08T12:38:36.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.076+0000 7ff7f6ffd700 1 -- 172.21.15.12:0/3485861883 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff7f8020090 con 0x7ff8000c3ce0 2023-07-08T12:38:36.078 INFO:teuthology.orchestra.run.smithi012.stdout:55834574858 2023-07-08T12:38:36.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.076+0000 7fb9a258a700 1 -- 172.21.15.12:0/2461141928 shutdown_connections 2023-07-08T12:38:36.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.076+0000 7fb9a258a700 1 -- 172.21.15.12:0/2461141928 wait complete. 2023-07-08T12:38:36.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.075+0000 7f2003440700 1 -- 172.21.15.12:0/2554776118 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1fdc0055a0 con 0x7f1ffc0c27b0 2023-07-08T12:38:36.079 INFO:teuthology.orchestra.run.smithi012.stdout:55834574858 2023-07-08T12:38:36.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7f56b404f700 1 -- 172.21.15.12:0/1930686030 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f56940055a0 con 0x7f56ac089570 2023-07-08T12:38:36.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7f1ff1ffb700 1 -- 172.21.15.12:0/2554776118 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1ff800bcf0 con 0x7f1ffc0c27b0 2023-07-08T12:38:36.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7fae2d21a700 1 -- 172.21.15.12:0/2010836567 shutdown_connections 2023-07-08T12:38:36.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7fae2d21a700 1 --2- 172.21.15.12:0/2010836567 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fae1406ad30 0x7fae1406d1e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7fae2d21a700 1 --2- 172.21.15.12:0/2010836567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae280c7af0 0x7fae280c7f60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7f41876aa700 1 -- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f416406ad30 msgr2=0x7f416406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7f41876aa700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f416406ad30 0x7f416406d1e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f41800c8990 tx=0x7f4170007e00 comp rx=0 tx=0).stop 2023-07-08T12:38:36.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7fae2d21a700 1 --2- 172.21.15.12:0/2010836567 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae10006720 0x7fae2815a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7fae2d21a700 1 --2- 172.21.15.12:0/2010836567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae28152820 0x7fae281ed340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7fae2d21a700 1 -- 172.21.15.12:0/2010836567 >> 172.21.15.12:0/2010836567 conn(0x7fae2806abc0 msgr2=0x7fae28152d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:36.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7f41876aa700 1 -- 172.21.15.12:0/41030716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c003a00 msgr2=0x7f4180158340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7f41876aa700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c003a00 0x7f4180158340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f417c014040 tx=0x7f417c001930 comp rx=0 tx=0).stop 2023-07-08T12:38:36.084 INFO:teuthology.orchestra.run.smithi012.stdout:55834574858 2023-07-08T12:38:36.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7f41876aa700 1 -- 172.21.15.12:0/41030716 shutdown_connections 2023-07-08T12:38:36.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7f41876aa700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f416406ad30 0x7f416406d1e0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f41800c8990 tx=0x7f4170007e00 comp rx=0 tx=0).stop 2023-07-08T12:38:36.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7f41876aa700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f418006cbb0 0x7f41800af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7f41876aa700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f416c003a00 0x7f4180158340 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7f41876aa700 1 --2- 172.21.15.12:0/41030716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f418006d550 0x7f4180155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.080+0000 7f41876aa700 1 -- 172.21.15.12:0/41030716 >> 172.21.15.12:0/41030716 conn(0x7f418006abc0 msgr2=0x7f418015a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:36.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.083+0000 7ff8082be700 1 -- 172.21.15.12:0/3485861883 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff7e406ad30 msgr2=0x7ff7e406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.083+0000 7ff8082be700 1 --2- 172.21.15.12:0/3485861883 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff7e406ad30 0x7ff7e406d1e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff80014dcf0 tx=0x7ff7fc009190 comp rx=0 tx=0).stop 2023-07-08T12:38:36.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.083+0000 7ff8082be700 1 -- 172.21.15.12:0/3485861883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8000c3ce0 msgr2=0x7ff80014d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.083+0000 7ff8082be700 1 --2- 172.21.15.12:0/3485861883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8000c3ce0 0x7ff80014d280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff8000c4220 tx=0x7ff7f8004ad0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.083+0000 7f56a27fc700 1 -- 172.21.15.12:0/1930686030 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f56a8057f80 con 0x7f56ac089570 2023-07-08T12:38:36.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.084+0000 7f2003440700 1 -- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1fe006ad30 msgr2=0x7f1fe006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.084+0000 7f2003440700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1fe006ad30 0x7f1fe006d1e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0b2760 tx=0x7f1fec008040 comp rx=0 tx=0).stop 2023-07-08T12:38:36.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.084+0000 7f2003440700 1 -- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ffc0c27b0 msgr2=0x7f1ffc162f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.084+0000 7f2003440700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ffc0c27b0 0x7f1ffc162f20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009c40 tx=0x7f1ff8004750 comp rx=0 tx=0).stop 2023-07-08T12:38:36.088 INFO:teuthology.orchestra.run.smithi012.stdout:55834574857 2023-07-08T12:38:36.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.084+0000 7f2003440700 1 -- 172.21.15.12:0/2554776118 shutdown_connections 2023-07-08T12:38:36.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.084+0000 7f2003440700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1fe006ad30 0x7f1fe006d1e0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0b2760 tx=0x7f1fec008040 comp rx=0 tx=0).stop 2023-07-08T12:38:36.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.084+0000 7f2003440700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ffc160700 0x7f1ffc14ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.084+0000 7f2003440700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ffc0c27b0 0x7f1ffc162f20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.084+0000 7f2003440700 1 --2- 172.21.15.12:0/2554776118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ffc0ccda0 0x7f1ffc161cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.084+0000 7f2003440700 1 -- 172.21.15.12:0/2554776118 >> 172.21.15.12:0/2554776118 conn(0x7f1ffc0cc290 msgr2=0x7f1ffc0ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:36.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.085+0000 7f41876aa700 1 -- 172.21.15.12:0/41030716 shutdown_connections 2023-07-08T12:38:36.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.086+0000 7f56b404f700 1 -- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f569006ace0 msgr2=0x7f569006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.086+0000 7f56b404f700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f569006ace0 0x7f569006d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0b48c0 tx=0x7f569c009040 comp rx=0 tx=0).stop 2023-07-08T12:38:36.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.086+0000 7f56b404f700 1 -- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56ac089570 msgr2=0x7f56ac06d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.086+0000 7f56b404f700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56ac089570 0x7f56ac06d4d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006e70 tx=0x7f56a800dd10 comp rx=0 tx=0).stop 2023-07-08T12:38:36.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.087+0000 7ff8082be700 1 -- 172.21.15.12:0/3485861883 shutdown_connections 2023-07-08T12:38:36.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.087+0000 7ff8082be700 1 --2- 172.21.15.12:0/3485861883 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff7e406ad30 0x7ff7e406d1e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.087+0000 7ff8082be700 1 --2- 172.21.15.12:0/3485861883 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff800160a10 0x7ff80014f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.087+0000 7ff8082be700 1 --2- 172.21.15.12:0/3485861883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8000c3ce0 0x7ff80014d280 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.087+0000 7ff8082be700 1 --2- 172.21.15.12:0/3485861883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800160640 0x7ff80006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.087+0000 7ff8082be700 1 -- 172.21.15.12:0/3485861883 >> 172.21.15.12:0/3485861883 conn(0x7ff80006ab60 msgr2=0x7ff8000d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:36.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.087+0000 7ff8082be700 1 -- 172.21.15.12:0/3485861883 shutdown_connections 2023-07-08T12:38:36.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.087+0000 7ff8082be700 1 -- 172.21.15.12:0/3485861883 wait complete. 2023-07-08T12:38:36.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.088+0000 7f56b404f700 1 -- 172.21.15.12:0/1930686030 shutdown_connections 2023-07-08T12:38:36.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.088+0000 7f56b404f700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f569006ace0 0x7f569006d190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.088+0000 7f56b404f700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56ac16c7e0 0x7f56ac0c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.088+0000 7f56b404f700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56ac089570 0x7f56ac06d4d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.088+0000 7f56b404f700 1 --2- 172.21.15.12:0/1930686030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ac0cb8a0 0x7f56ac0c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.088+0000 7f56b404f700 1 -- 172.21.15.12:0/1930686030 >> 172.21.15.12:0/1930686030 conn(0x7f56ac06abc0 msgr2=0x7f56ac06e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:36.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.088+0000 7f41876aa700 1 -- 172.21.15.12:0/41030716 wait complete. 2023-07-08T12:38:36.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.088+0000 7f2003440700 1 -- 172.21.15.12:0/2554776118 shutdown_connections 2023-07-08T12:38:36.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.088+0000 7f2003440700 1 -- 172.21.15.12:0/2554776118 wait complete. 2023-07-08T12:38:36.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.090+0000 7f56b404f700 1 -- 172.21.15.12:0/1930686030 shutdown_connections 2023-07-08T12:38:36.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.090+0000 7f56b404f700 1 -- 172.21.15.12:0/1930686030 wait complete. 2023-07-08T12:38:36.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.085+0000 7fae2d21a700 1 -- 172.21.15.12:0/2010836567 shutdown_connections 2023-07-08T12:38:36.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.085+0000 7fae2d21a700 1 -- 172.21.15.12:0/2010836567 wait complete. 2023-07-08T12:38:36.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.093+0000 7f90210c7700 1 -- 172.21.15.12:0/2562869102 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f901c04ea60 con 0x7f901c0b1c70 2023-07-08T12:38:36.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.094+0000 7f90037fe700 1 -- 172.21.15.12:0/2562869102 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9010057bf0 con 0x7f901c0b1c70 2023-07-08T12:38:36.095 INFO:teuthology.orchestra.run.smithi012.stdout:55834574858 2023-07-08T12:38:36.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.1 2023-07-08T12:38:36.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.098+0000 7f90210c7700 1 -- 172.21.15.12:0/2562869102 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f900806ace0 msgr2=0x7f900806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.098+0000 7f90210c7700 1 --2- 172.21.15.12:0/2562869102 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f900806ace0 0x7f900806d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f901c16a340 tx=0x7f900c006c90 comp rx=0 tx=0).stop 2023-07-08T12:38:36.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.098+0000 7f90210c7700 1 -- 172.21.15.12:0/2562869102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f901c0b1c70 msgr2=0x7f901c0c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.098+0000 7f90210c7700 1 --2- 172.21.15.12:0/2562869102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f901c0b1c70 0x7f901c0c35b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f901000b550 tx=0x7f901000b910 comp rx=0 tx=0).stop 2023-07-08T12:38:36.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.108+0000 7f90210c7700 1 -- 172.21.15.12:0/2562869102 shutdown_connections 2023-07-08T12:38:36.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.108+0000 7f90210c7700 1 --2- 172.21.15.12:0/2562869102 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f900806ace0 0x7f900806d190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.108+0000 7f90210c7700 1 --2- 172.21.15.12:0/2562869102 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f901c0c94f0 0x7f901c1f9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.108+0000 7f90210c7700 1 --2- 172.21.15.12:0/2562869102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f901c0b1c70 0x7f901c0c35b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.108+0000 7f90210c7700 1 --2- 172.21.15.12:0/2562869102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f901c0ce8c0 0x7f901c0c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.108+0000 7f90210c7700 1 -- 172.21.15.12:0/2562869102 >> 172.21.15.12:0/2562869102 conn(0x7f901c0cc290 msgr2=0x7f901c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:36.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.109+0000 7f90210c7700 1 -- 172.21.15.12:0/2562869102 shutdown_connections 2023-07-08T12:38:36.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.109+0000 7f90210c7700 1 -- 172.21.15.12:0/2562869102 wait complete. 2023-07-08T12:38:36.112 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.6 2023-07-08T12:38:36.116 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.3 2023-07-08T12:38:36.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.115+0000 7f4c2672f700 1 -- 172.21.15.12:0/2192108546 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4bfc0055a0 con 0x7f4c200c9970 2023-07-08T12:38:36.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.5 2023-07-08T12:38:36.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.4 2023-07-08T12:38:36.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.116+0000 7f4c1cff9700 1 -- 172.21.15.12:0/2192108546 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4c1001e020 con 0x7f4c200c9970 2023-07-08T12:38:36.120 INFO:teuthology.orchestra.run.smithi012.stdout:55834574859 2023-07-08T12:38:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.118+0000 7f4c2672f700 1 -- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4c0406ad80 msgr2=0x7f4c0406d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.118+0000 7f4c2672f700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4c0406ad80 0x7f4c0406d230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009b90 tx=0x7f4c14008030 comp rx=0 tx=0).stop 2023-07-08T12:38:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.118+0000 7f4c2672f700 1 -- 172.21.15.12:0/2192108546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c200c9970 msgr2=0x7f4c2014eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.118+0000 7f4c2672f700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c200c9970 0x7f4c2014eff0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000d900 tx=0x7f4c1000dcc0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.118+0000 7f4c2672f700 1 -- 172.21.15.12:0/2192108546 shutdown_connections 2023-07-08T12:38:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.118+0000 7f4c2672f700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4c0406ad80 0x7f4c0406d230 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.118+0000 7f4c2672f700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c200cb6e0 0x7f4c200aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.118+0000 7f4c2672f700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c200c9970 0x7f4c2014eff0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.118+0000 7f4c2672f700 1 --2- 172.21.15.12:0/2192108546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c200c9d40 0x7f4c200c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.118+0000 7f4c2672f700 1 -- 172.21.15.12:0/2192108546 >> 172.21.15.12:0/2192108546 conn(0x7f4c2006abc0 msgr2=0x7f4c200ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.118+0000 7f4c2672f700 1 -- 172.21.15.12:0/2192108546 shutdown_connections 2023-07-08T12:38:36.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:36.119+0000 7f4c2672f700 1 -- 172.21.15.12:0/2192108546 wait complete. 2023-07-08T12:38:36.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.0 2023-07-08T12:38:36.127 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.2 2023-07-08T12:38:36.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574859 for osd.7 2023-07-08T12:38:37.100 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-08T12:38:37.113 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-08T12:38:37.119 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-08T12:38:37.119 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-08T12:38:37.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-08T12:38:37.125 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-08T12:38:37.128 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-08T12:38:37.134 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-08T12:38:37.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.310+0000 7fd930222700 1 Processor -- start 2023-07-08T12:38:37.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.311+0000 7fd930222700 1 -- start start 2023-07-08T12:38:37.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.312+0000 7fd930222700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9280c9970 0x7fd9280cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.312+0000 7fd930222700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9280cce20 con 0x7fd9280c9d40 2023-07-08T12:38:37.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.312+0000 7fd930222700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9280cd100 con 0x7fd9280cb8b0 2023-07-08T12:38:37.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.312+0000 7fd930222700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280cd3e0 con 0x7fd9280c9970 2023-07-08T12:38:37.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.313+0000 7fd92d7bd700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd9280c9d40 0x7fd9280cb1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37276/0 (socket says 172.21.15.12:37276) 2023-07-08T12:38:37.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.313+0000 7fd92d7bd700 1 -- 172.21.15.12:0/2212705080 learned_addr learned my addr 172.21.15.12:0/2212705080 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:37.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.314+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026903345 0 0) 0x7fd9280cce20 con 0x7fd9280c9d40 2023-07-08T12:38:37.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.314+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9140035b0 con 0x7fd9280c9d40 2023-07-08T12:38:37.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.314+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 177300425 0 0) 0x7fd9140035b0 con 0x7fd9280c9d40 2023-07-08T12:38:37.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.314+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9280c9970 msgr2=0x7fd9280cdec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-08T12:38:37.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.315+0000 7fd92cfbc700 1 --2- 172.21.15.12:0/2212705080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9280c9970 0x7fd9280cdec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.315+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 >> v1:172.21.15.160:6789/0 conn(0x7fd9280cb8b0 legacy=0x7fd928157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.315+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280cd6c0 con 0x7fd9280c9d40 2023-07-08T12:38:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.315+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7fd918001670 con 0x7fd9280c9d40 2023-07-08T12:38:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.315+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 >> v1:172.21.15.12:6789/0 conn(0x7fd9280c9d40 legacy=0x7fd9280cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.315+0000 7fd92cfbc700 1 --2- 172.21.15.12:0/2212705080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd914003a30 0x7fd914003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.315+0000 7fd92cfbc700 1 --2- 172.21.15.12:0/2212705080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd914004850 0x7fd914004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.315+0000 7fd92cfbc700 1 --2- 172.21.15.12:0/2212705080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd914005250 0x7fd91400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.315+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280cd3e0 con 0x7fd914004850 2023-07-08T12:38:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.315+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91400c160 con 0x7fd914005250 2023-07-08T12:38:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.315+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91400c320 con 0x7fd914003a30 2023-07-08T12:38:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.315+0000 7fd92d7bd700 1 --2- 172.21.15.12:0/2212705080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd914004850 0x7fd914004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.316+0000 7fd92d7bd700 1 -- 172.21.15.12:0/2212705080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd914003a30 msgr2=0x7fd914003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-08T12:38:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.316+0000 7fd92d7bd700 1 --2- 172.21.15.12:0/2212705080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd914003a30 0x7fd914003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.316+0000 7fd92d7bd700 1 -- 172.21.15.12:0/2212705080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd914005250 msgr2=0x7fd91400ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-08T12:38:37.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.316+0000 7fd92d7bd700 1 --2- 172.21.15.12:0/2212705080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd914005250 0x7fd91400ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.316+0000 7fd92d7bd700 1 -- 172.21.15.12:0/2212705080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280cd6c0 con 0x7fd914004850 2023-07-08T12:38:37.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.316+0000 7fd92d7bd700 1 --2- 172.21.15.12:0/2212705080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd914004850 0x7fd914004cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd91800d3a0 tx=0x7fd914005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f93e87db0bac500 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.317+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91800f400 con 0x7fd914004850 2023-07-08T12:38:37.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.317+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91800f560 con 0x7fd914004850 2023-07-08T12:38:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.317+0000 7fd92cfbc700 1 -- 172.21.15.12:0/2212705080 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91800f8f0 con 0x7fd914004850 2023-07-08T12:38:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.318+0000 7fd930222700 1 -- 172.21.15.12:0/2212705080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd914004850 msgr2=0x7fd914004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.318+0000 7fd930222700 1 --2- 172.21.15.12:0/2212705080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd914004850 0x7fd914004cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd91800d3a0 tx=0x7fd914005880 comp rx=0 tx=0).stop 2023-07-08T12:38:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.318+0000 7fd92e7bf700 1 -- 172.21.15.12:0/2212705080 reap_dead start 2023-07-08T12:38:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.319+0000 7fd930222700 1 -- 172.21.15.12:0/2212705080 shutdown_connections 2023-07-08T12:38:37.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.319+0000 7fd930222700 1 -- 172.21.15.12:0/2212705080 >> 172.21.15.12:0/2212705080 conn(0x7fd92806abc0 msgr2=0x7fd9280b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.319+0000 7fd930222700 1 -- 172.21.15.12:0/2212705080 shutdown_connections 2023-07-08T12:38:37.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.319+0000 7fd930222700 1 -- 172.21.15.12:0/2212705080 wait complete. 2023-07-08T12:38:37.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.321+0000 7fd930222700 1 Processor -- start 2023-07-08T12:38:37.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.322+0000 7fd930222700 1 -- start start 2023-07-08T12:38:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.323+0000 7fd930222700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd914004850 0x7fd9280c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.323+0000 7fd930222700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9281f1ed0 0x7fd9280c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.323+0000 7fd930222700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9280c5fe0 0x7fd9280c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.323+0000 7fd930222700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280c6970 con 0x7fd9280c5fe0 2023-07-08T12:38:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.323+0000 7fd930222700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280c8030 con 0x7fd914004850 2023-07-08T12:38:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.323+0000 7fd930222700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280c8310 con 0x7fd9281f1ed0 2023-07-08T12:38:37.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.324+0000 7fd92e7bf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9280c5fe0 0x7fd9280c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.324+0000 7fd92e7bf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9280c5fe0 0x7fd9280c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49888/0 (socket says 172.21.15.12:49888) 2023-07-08T12:38:37.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.324+0000 7fd92e7bf700 1 -- 172.21.15.12:0/1302984672 learned_addr learned my addr 172.21.15.12:0/1302984672 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:37.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.324+0000 7fd92e7bf700 1 -- 172.21.15.12:0/1302984672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9281f1ed0 msgr2=0x7fd9280c4500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-08T12:38:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.324+0000 7fd92e7bf700 1 --2- 172.21.15.12:0/1302984672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9281f1ed0 0x7fd9280c4500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.324+0000 7fd92e7bf700 1 -- 172.21.15.12:0/1302984672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd914004850 msgr2=0x7fd9280c5890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-08T12:38:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.324+0000 7fd92e7bf700 1 --2- 172.21.15.12:0/1302984672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd914004850 0x7fd9280c5890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.324+0000 7fd92e7bf700 1 -- 172.21.15.12:0/1302984672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280c85f0 con 0x7fd9280c5fe0 2023-07-08T12:38:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.325+0000 7fd92e7bf700 1 --2- 172.21.15.12:0/1302984672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9280c5fe0 0x7fd9280c6430 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd9280cbc80 tx=0x7fd92000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.327+0000 7fd91e7fc700 1 -- 172.21.15.12:0/1302984672 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92000a520 con 0x7fd9280c5fe0 2023-07-08T12:38:37.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.327+0000 7fd91e7fc700 1 -- 172.21.15.12:0/1302984672 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92000a680 con 0x7fd9280c5fe0 2023-07-08T12:38:37.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.327+0000 7fd91e7fc700 1 -- 172.21.15.12:0/1302984672 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92000aa10 con 0x7fd9280c5fe0 2023-07-08T12:38:37.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.327+0000 7fd930222700 1 -- 172.21.15.12:0/1302984672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9280c88d0 con 0x7fd9280c5fe0 2023-07-08T12:38:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.327+0000 7fd930222700 1 -- 172.21.15.12:0/1302984672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9281f2340 con 0x7fd9280c5fe0 2023-07-08T12:38:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.332+0000 7fd91e7fc700 1 -- 172.21.15.12:0/1302984672 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd92000ab70 con 0x7fd9280c5fe0 2023-07-08T12:38:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.333+0000 7fd91e7fc700 1 --2- 172.21.15.12:0/1302984672 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd90c06ad80 0x7fd90c06d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.333+0000 7fd91e7fc700 1 -- 172.21.15.12:0/1302984672 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd920088e70 con 0x7fd9280c5fe0 2023-07-08T12:38:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.334+0000 7fd930222700 1 -- 172.21.15.12:0/1302984672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9100052f0 con 0x7fd9280c5fe0 2023-07-08T12:38:37.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.336+0000 7fd92dfbe700 1 --2- 172.21.15.12:0/1302984672 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd90c06ad80 0x7fd90c06d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.337+0000 7fd92dfbe700 1 --2- 172.21.15.12:0/1302984672 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd90c06ad80 0x7fd90c06d230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd9280cc560 tx=0x7fd924006c60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.343+0000 7fd91e7fc700 1 -- 172.21.15.12:0/1302984672 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd920053ca0 con 0x7fd9280c5fe0 2023-07-08T12:38:37.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.371+0000 7f6586301700 1 Processor -- start 2023-07-08T12:38:37.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.371+0000 7f6586301700 1 -- start start 2023-07-08T12:38:37.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.373+0000 7f6586301700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f65800c9970 0x7f65800cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.373+0000 7f6586301700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65800c9d40 0x7f65800cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.373+0000 7f6586301700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65800cce20 con 0x7f65800cb6e0 2023-07-08T12:38:37.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.373+0000 7f6586301700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65800cd100 con 0x7f65800c9970 2023-07-08T12:38:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.373+0000 7f6586301700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65800cd3e0 con 0x7f65800c9d40 2023-07-08T12:38:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.373+0000 7f657ffff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f65800c9970 0x7f65800cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.373+0000 7f657ffff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f65800c9970 0x7f65800cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49896/0 (socket says 172.21.15.12:49896) 2023-07-08T12:38:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.373+0000 7f657ffff700 1 -- 172.21.15.12:0/1905563655 learned_addr learned my addr 172.21.15.12:0/1905563655 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.374+0000 7f657ffff700 1 -- 172.21.15.12:0/1905563655 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65800c9d40 msgr2=0x7f65800cb1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-08T12:38:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.374+0000 7f657ffff700 1 --2- 172.21.15.12:0/1905563655 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65800c9d40 0x7f65800cb1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.374+0000 7f657ffff700 1 -- 172.21.15.12:0/1905563655 >> v1:172.21.15.12:6789/0 conn(0x7f65800cb6e0 legacy=0x7f6580157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:37.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.374+0000 7f657ffff700 1 -- 172.21.15.12:0/1905563655 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65800cd6c0 con 0x7f65800c9970 2023-07-08T12:38:37.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.374+0000 7f657ffff700 1 --2- 172.21.15.12:0/1905563655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65800c9970 0x7f65800cdec0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f65740088f0 tx=0x7f657400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da491926811c48f server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.377+0000 7f8332a41700 1 Processor -- start 2023-07-08T12:38:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.377+0000 7f91f8303700 1 Processor -- start 2023-07-08T12:38:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.377+0000 7f91f8303700 1 -- start start 2023-07-08T12:38:37.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.377+0000 7f8332a41700 1 -- start start 2023-07-08T12:38:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.378+0000 7f8332a41700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f832c06a640 0x7f832c070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.378+0000 7f8332a41700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c06b5f0 0x7f832c161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.378+0000 7f8332a41700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c0ce9c0 con 0x7f832c0c2330 2023-07-08T12:38:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.378+0000 7f8332a41700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c161b80 con 0x7f832c06a640 2023-07-08T12:38:37.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.378+0000 7f8332a41700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0ccb50 con 0x7f832c06b5f0 2023-07-08T12:38:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.378+0000 7f91f8303700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f91f0160700 0x7f91f0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.378+0000 7f91f8303700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f91f00c27b0 0x7f91f006a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.378+0000 7f91f8303700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f0160ad0 con 0x7f91f00ccda0 2023-07-08T12:38:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.378+0000 7f91f8303700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f006ff20 con 0x7f91f00c27b0 2023-07-08T12:38:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.378+0000 7f91f8303700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f00cd560 con 0x7f91f0160700 2023-07-08T12:38:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f91f609f700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f91f0160700 0x7f91f0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f91f68a0700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f91f00ccda0 0x7f91f00b1290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37304/0 (socket says 172.21.15.12:37304) 2023-07-08T12:38:37.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f91f68a0700 1 -- 172.21.15.12:0/2473247703 learned_addr learned my addr 172.21.15.12:0/2473247703 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:37.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832b7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f832c06a640 0x7f832c070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832bfff700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f832c0c2330 0x7f832c069f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:46516/0 (socket says 172.21.15.12:46516) 2023-07-08T12:38:37.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832bfff700 1 -- 172.21.15.12:0/4091549309 learned_addr learned my addr 172.21.15.12:0/4091549309 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.378+0000 7f657effd700 1 -- 172.21.15.12:0/1905563655 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574010070 con 0x7f65800c9970 2023-07-08T12:38:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f657effd700 1 -- 172.21.15.12:0/1905563655 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657400da40 con 0x7f65800c9970 2023-07-08T12:38:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f657effd700 1 -- 172.21.15.12:0/1905563655 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657400ddd0 con 0x7f65800c9970 2023-07-08T12:38:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655902448 0 0) 0x7f832c0ce9c0 con 0x7f832c0c2330 2023-07-08T12:38:37.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83140035b0 con 0x7f832c0c2330 2023-07-08T12:38:37.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f91f609f700 1 -- 172.21.15.12:0/2473247703 >> v2:172.21.15.12:3300/0 conn(0x7f91f00c27b0 msgr2=0x7f91f006a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:37.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f91f609f700 1 --2- 172.21.15.12:0/2473247703 >> v2:172.21.15.12:3300/0 conn(0x7f91f00c27b0 0x7f91f006a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f91f609f700 1 -- 172.21.15.12:0/2473247703 >> v1:172.21.15.12:6789/0 conn(0x7f91f00ccda0 legacy=0x7f91f00b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f91f609f700 1 -- 172.21.15.12:0/2473247703 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f0164c50 con 0x7f91f0160700 2023-07-08T12:38:37.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f6586301700 1 -- 172.21.15.12:0/1905563655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65800c9970 msgr2=0x7f65800cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f6586301700 1 --2- 172.21.15.12:0/1905563655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65800c9970 0x7f65800cdec0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f65740088f0 tx=0x7f657400d330 comp rx=0 tx=0).stop 2023-07-08T12:38:37.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f6586301700 1 -- 172.21.15.12:0/1905563655 shutdown_connections 2023-07-08T12:38:37.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f6586301700 1 --2- 172.21.15.12:0/1905563655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65800c9970 0x7f65800cdec0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f6586301700 1 --2- 172.21.15.12:0/1905563655 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65800c9d40 0x7f65800cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f6586301700 1 -- 172.21.15.12:0/1905563655 >> 172.21.15.12:0/1905563655 conn(0x7f658006abc0 msgr2=0x7f65800b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f8330fde700 1 --2- 172.21.15.12:0/4091549309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c06b5f0 0x7f832c161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 <== mon.1 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3523630632 0 0) 0x7f83140035b0 con 0x7f832c0c2330 2023-07-08T12:38:37.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c06b5f0 msgr2=0x7f832c161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:37.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 --2- 172.21.15.12:0/4091549309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c06b5f0 0x7f832c161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 >> v2:172.21.15.12:3300/0 conn(0x7f832c06a640 msgr2=0x7f832c070bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:37.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 --2- 172.21.15.12:0/4091549309 >> v2:172.21.15.12:3300/0 conn(0x7f832c06a640 0x7f832c070bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0cce30 con 0x7f832c0c2330 2023-07-08T12:38:37.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 <== mon.1 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f8320001670 con 0x7f832c0c2330 2023-07-08T12:38:37.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 >> v1:172.21.15.160:6789/0 conn(0x7f832c0c2330 legacy=0x7f832c069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 --2- 172.21.15.12:0/4091549309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8314003a00 0x7f8314003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 --2- 172.21.15.12:0/4091549309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83140043e0 0x7f83140048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 --2- 172.21.15.12:0/4091549309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8314005070 0x7f831400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c161b80 con 0x7f83140043e0 2023-07-08T12:38:37.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0ccb50 con 0x7f8314005070 2023-07-08T12:38:37.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.379+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831400c000 con 0x7f8314003a00 2023-07-08T12:38:37.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f832bfff700 1 --2- 172.21.15.12:0/4091549309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8314003a00 0x7f8314003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f6586301700 1 -- 172.21.15.12:0/1905563655 shutdown_connections 2023-07-08T12:38:37.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f6586301700 1 -- 172.21.15.12:0/1905563655 wait complete. 2023-07-08T12:38:37.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f8330fde700 1 --2- 172.21.15.12:0/4091549309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8314005070 0x7f831400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f832bfff700 1 -- 172.21.15.12:0/4091549309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8314005070 msgr2=0x7f831400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:37.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f832bfff700 1 --2- 172.21.15.12:0/4091549309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8314005070 0x7f831400ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f832bfff700 1 -- 172.21.15.12:0/4091549309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83140043e0 msgr2=0x7f83140048b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-08T12:38:37.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f832bfff700 1 --2- 172.21.15.12:0/4091549309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83140043e0 0x7f83140048b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f832bfff700 1 -- 172.21.15.12:0/4091549309 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0cce30 con 0x7f8314003a00 2023-07-08T12:38:37.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.380+0000 7f832bfff700 1 --2- 172.21.15.12:0/4091549309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8314003a00 0x7f8314003e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f832000c2f0 tx=0x7f8314005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d5adbe75acc5d5d server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.381+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320007c80 con 0x7f8314003a00 2023-07-08T12:38:37.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.381+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832000ce70 con 0x7f8314003a00 2023-07-08T12:38:37.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.381+0000 7f832affd700 1 -- 172.21.15.12:0/4091549309 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320005b80 con 0x7f8314003a00 2023-07-08T12:38:37.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.382+0000 7f6586301700 1 Processor -- start 2023-07-08T12:38:37.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.382+0000 7f8332a41700 1 -- 172.21.15.12:0/4091549309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8314003a00 msgr2=0x7f8314003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.382+0000 7f8332a41700 1 --2- 172.21.15.12:0/4091549309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8314003a00 0x7f8314003e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f832000c2f0 tx=0x7f8314005880 comp rx=0 tx=0).stop 2023-07-08T12:38:37.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.382+0000 7f8330fde700 1 -- 172.21.15.12:0/4091549309 reap_dead start 2023-07-08T12:38:37.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.382+0000 7f8332a41700 1 -- 172.21.15.12:0/4091549309 shutdown_connections 2023-07-08T12:38:37.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.382+0000 7f8332a41700 1 -- 172.21.15.12:0/4091549309 >> 172.21.15.12:0/4091549309 conn(0x7f832c0cb9f0 msgr2=0x7f832c0cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.382+0000 7f6586301700 1 -- start start 2023-07-08T12:38:37.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f6586301700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65800c9970 0x7f65800c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f6586301700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65800c9d40 0x7f65801fa780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f6586301700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65800cb6e0 0x7f65801f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f6586301700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65800c49d0 con 0x7f65800c9d40 2023-07-08T12:38:37.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f6586301700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65801f7790 con 0x7f65800c9970 2023-07-08T12:38:37.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f6586301700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65801f71d0 con 0x7f65800cb6e0 2023-07-08T12:38:37.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f657f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65800c9d40 0x7f65801fa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f657f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65800c9d40 0x7f65801fa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49916/0 (socket says 172.21.15.12:49916) 2023-07-08T12:38:37.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f657f7fe700 1 -- 172.21.15.12:0/4173348686 learned_addr learned my addr 172.21.15.12:0/4173348686 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:37.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f8332a41700 1 -- 172.21.15.12:0/4091549309 shutdown_connections 2023-07-08T12:38:37.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f658489e700 1 --2- 172.21.15.12:0/4173348686 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65800cb6e0 0x7f65801f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f657f7fe700 1 -- 172.21.15.12:0/4173348686 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65800cb6e0 msgr2=0x7f65801f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f657f7fe700 1 --2- 172.21.15.12:0/4173348686 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65800cb6e0 0x7f65801f6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f657f7fe700 1 -- 172.21.15.12:0/4173348686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65800c9970 msgr2=0x7f65800c4490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-08T12:38:37.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f657f7fe700 1 --2- 172.21.15.12:0/4173348686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65800c9970 0x7f65800c4490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.384+0000 7f657f7fe700 1 -- 172.21.15.12:0/4173348686 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65740085d0 con 0x7f65800c9d40 2023-07-08T12:38:37.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.384+0000 7f657f7fe700 1 --2- 172.21.15.12:0/4173348686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65800c9d40 0x7f65801fa780 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f657000b550 tx=0x7f657000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.385+0000 7f3cdb83b700 1 Processor -- start 2023-07-08T12:38:37.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.383+0000 7f8332a41700 1 -- 172.21.15.12:0/4091549309 wait complete. 2023-07-08T12:38:37.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.385+0000 7f8332a41700 1 Processor -- start 2023-07-08T12:38:37.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f657cff9700 1 -- 172.21.15.12:0/4173348686 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570011830 con 0x7f65800c9d40 2023-07-08T12:38:37.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f657cff9700 1 -- 172.21.15.12:0/4173348686 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65700119b0 con 0x7f65800c9d40 2023-07-08T12:38:37.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f657cff9700 1 -- 172.21.15.12:0/4173348686 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570011d60 con 0x7f65800c9d40 2023-07-08T12:38:37.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.385+0000 7f8332a41700 1 -- start start 2023-07-08T12:38:37.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f8332a41700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8314003a00 0x7f832c14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f8332a41700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c0b2f20 0x7f832c0b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f8332a41700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f832c0c2700 0x7f832c0c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f8332a41700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0b2b00 con 0x7f8314003a00 2023-07-08T12:38:37.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f8332a41700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c14f9e0 con 0x7f832c0c2700 2023-07-08T12:38:37.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f8332a41700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0c30b0 con 0x7f832c0b2f20 2023-07-08T12:38:37.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f832b7fe700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c0b2f20 0x7f832c0b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f8330fde700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f832c0c2700 0x7f832c0c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f832b7fe700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c0b2f20 0x7f832c0b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59540/0 (socket says 172.21.15.12:59540) 2023-07-08T12:38:37.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f832b7fe700 1 -- 172.21.15.12:0/214944575 learned_addr learned my addr 172.21.15.12:0/214944575 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f91f609f700 1 --2- 172.21.15.12:0/2473247703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f0160700 0x7f91f0069f30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f91f00cf770 tx=0x7f91e0009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=396cc85843f7f149 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f832b7fe700 1 -- 172.21.15.12:0/214944575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f832c0c2700 msgr2=0x7f832c0c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f832b7fe700 1 --2- 172.21.15.12:0/214944575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f832c0c2700 0x7f832c0c2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f832b7fe700 1 -- 172.21.15.12:0/214944575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8314003a00 msgr2=0x7f832c14f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f832b7fe700 1 --2- 172.21.15.12:0/214944575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8314003a00 0x7f832c14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f832b7fe700 1 -- 172.21.15.12:0/214944575 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0b1e80 con 0x7f832c0b2f20 2023-07-08T12:38:37.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.385+0000 7f3cdb83b700 1 -- start start 2023-07-08T12:38:37.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f3cdb83b700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3cd40c9970 0x7f3cd40cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f3cdb83b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd40c9d40 0x7f3cd40cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f3cdb83b700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd40cce20 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f3cdb83b700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd40cd100 con 0x7f3cd40c9970 2023-07-08T12:38:37.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f3cdb83b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40cd3e0 con 0x7f3cd40c9d40 2023-07-08T12:38:37.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f3cd8dd6700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd40c9d40 0x7f3cd40cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f3cd8dd6700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd40c9d40 0x7f3cd40cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59544/0 (socket says 172.21.15.12:59544) 2023-07-08T12:38:37.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f3cd8dd6700 1 -- 172.21.15.12:0/347333965 learned_addr learned my addr 172.21.15.12:0/347333965 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:37.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f832b7fe700 1 --2- 172.21.15.12:0/214944575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c0b2f20 0x7f832c0b3390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f831c00e960 tx=0x7f831c00ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.386+0000 7f6586301700 1 -- 172.21.15.12:0/4173348686 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65801f74b0 con 0x7f65800c9d40 2023-07-08T12:38:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f3cd95d7700 1 --2- 172.21.15.12:0/347333965 >> v2:172.21.15.160:3300/0 conn(0x7f3cd40c9970 0x7f3cd40cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f3cd8dd6700 1 -- 172.21.15.12:0/347333965 >> v2:172.21.15.160:3300/0 conn(0x7f3cd40c9970 msgr2=0x7f3cd40cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f3cd8dd6700 1 --2- 172.21.15.12:0/347333965 >> v2:172.21.15.160:3300/0 conn(0x7f3cd40c9970 0x7f3cd40cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f3cd8dd6700 1 -- 172.21.15.12:0/347333965 >> v1:172.21.15.12:6789/0 conn(0x7f3cd40cb6e0 legacy=0x7f3cd4157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:37.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f3cd8dd6700 1 -- 172.21.15.12:0/347333965 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd40cd6c0 con 0x7f3cd40c9d40 2023-07-08T12:38:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f8328ff9700 1 -- 172.21.15.12:0/214944575 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c0096d0 con 0x7f832c0b2f20 2023-07-08T12:38:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f8328ff9700 1 -- 172.21.15.12:0/214944575 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c00eeb0 con 0x7f832c0b2f20 2023-07-08T12:38:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f8328ff9700 1 -- 172.21.15.12:0/214944575 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00ca50 con 0x7f832c0b2f20 2023-07-08T12:38:37.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f8332a41700 1 -- 172.21.15.12:0/214944575 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c0b2160 con 0x7f832c0b2f20 2023-07-08T12:38:37.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f3cd8dd6700 1 --2- 172.21.15.12:0/347333965 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd40c9d40 0x7f3cd40cb1a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40092e0 tx=0x7f3cc40095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ad471e2d7e4701 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f8332a41700 1 -- 172.21.15.12:0/214944575 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c0b2720 con 0x7f832c0b2f20 2023-07-08T12:38:37.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f91f509d700 1 -- 172.21.15.12:0/2473247703 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0011830 con 0x7f91f0160700 2023-07-08T12:38:37.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f91f509d700 1 -- 172.21.15.12:0/2473247703 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e0011990 con 0x7f91f0160700 2023-07-08T12:38:37.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f91f509d700 1 -- 172.21.15.12:0/2473247703 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0011d20 con 0x7f91f0160700 2023-07-08T12:38:37.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.388+0000 7f3ccbfff700 1 -- 172.21.15.12:0/347333965 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4004230 con 0x7f3cd40c9d40 2023-07-08T12:38:37.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.388+0000 7f3ccbfff700 1 -- 172.21.15.12:0/347333965 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc4004390 con 0x7f3cd40c9d40 2023-07-08T12:38:37.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.388+0000 7f3ccbfff700 1 -- 172.21.15.12:0/347333965 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400c770 con 0x7f3cd40c9d40 2023-07-08T12:38:37.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f3e84d2c700 1 Processor -- start 2023-07-08T12:38:37.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.387+0000 7f3e84d2c700 1 -- start start 2023-07-08T12:38:37.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.388+0000 7f6586301700 1 -- 172.21.15.12:0/4173348686 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65801fa260 con 0x7f65800c9d40 2023-07-08T12:38:37.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.388+0000 7f3e84d2c700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3e800cb8a0 0x7f3e8016c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.388+0000 7f3e84d2c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3e80089570 0x7f3e800ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.388+0000 7f3e84d2c700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e800ce620 con 0x7f3e800cda10 2023-07-08T12:38:37.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.388+0000 7f3e84d2c700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e80089e70 con 0x7f3e80089570 2023-07-08T12:38:37.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.389+0000 7f3e84d2c700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e800caac0 con 0x7f3e800cb8a0 2023-07-08T12:38:37.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.389+0000 7f3e7e59c700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f3e800cda10 0x7f3e800cb190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:46550/0 (socket says 172.21.15.12:46550) 2023-07-08T12:38:37.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.389+0000 7f3e7e59c700 1 -- 172.21.15.12:0/3992339042 learned_addr learned my addr 172.21.15.12:0/3992339042 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:37.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.389+0000 7f3e7dd9b700 1 --2- 172.21.15.12:0/3992339042 >> v2:172.21.15.160:3300/0 conn(0x7f3e800cb8a0 0x7f3e8016c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.389+0000 7f3e7dd9b700 1 -- 172.21.15.12:0/3992339042 >> v2:172.21.15.12:3300/0 conn(0x7f3e80089570 msgr2=0x7f3e800ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:37.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.389+0000 7f3e7dd9b700 1 --2- 172.21.15.12:0/3992339042 >> v2:172.21.15.12:3300/0 conn(0x7f3e80089570 0x7f3e800ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.389+0000 7f3e7dd9b700 1 -- 172.21.15.12:0/3992339042 >> v1:172.21.15.160:6789/0 conn(0x7f3e800cda10 legacy=0x7f3e800cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.389+0000 7f3cdb83b700 1 -- 172.21.15.12:0/347333965 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd40c9d40 msgr2=0x7f3cd40cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.389+0000 7f3cdb83b700 1 --2- 172.21.15.12:0/347333965 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd40c9d40 0x7f3cd40cb1a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40092e0 tx=0x7f3cc40095f0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.389+0000 7f3e7d59a700 1 -- 172.21.15.12:0/3992339042 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840629628 0 0) 0x7f3e70001110 con 0x7f3e800cda10 2023-07-08T12:38:37.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.389+0000 7f3e7dd9b700 1 -- 172.21.15.12:0/3992339042 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e800ccc50 con 0x7f3e800cb8a0 2023-07-08T12:38:37.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.389+0000 7f3e7dd9b700 1 --2- 172.21.15.12:0/3992339042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e800cb8a0 0x7f3e8016c570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3e800ca730 tx=0x7f3e74009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f4f64ba6d7f51a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.388+0000 7f91f8303700 1 -- 172.21.15.12:0/2473247703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f0160700 msgr2=0x7f91f0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.388+0000 7f91f8303700 1 --2- 172.21.15.12:0/2473247703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f0160700 0x7f91f0069f30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f91f00cf770 tx=0x7f91e0009b20 comp rx=0 tx=0).stop 2023-07-08T12:38:37.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.389+0000 7f8332a41700 1 -- 172.21.15.12:0/214944575 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c0b2c40 con 0x7f832c0b2f20 2023-07-08T12:38:37.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.390+0000 7f8328ff9700 1 -- 172.21.15.12:0/214944575 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f831c00cbb0 con 0x7f832c0b2f20 2023-07-08T12:38:37.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.390+0000 7f3cdb83b700 1 -- 172.21.15.12:0/347333965 shutdown_connections 2023-07-08T12:38:37.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.390+0000 7f3cdb83b700 1 --2- 172.21.15.12:0/347333965 >> v2:172.21.15.160:3300/0 conn(0x7f3cd40c9970 0x7f3cd40cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.390+0000 7f3cdb83b700 1 --2- 172.21.15.12:0/347333965 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd40c9d40 0x7f3cd40cb1a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.390+0000 7f3cdb83b700 1 -- 172.21.15.12:0/347333965 >> 172.21.15.12:0/347333965 conn(0x7f3cd406abc0 msgr2=0x7f3cd40b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.390+0000 7f3cdb83b700 1 -- 172.21.15.12:0/347333965 shutdown_connections 2023-07-08T12:38:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.390+0000 7f8328ff9700 1 --2- 172.21.15.12:0/214944575 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f831806ace0 0x7f831806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.390+0000 7f8328ff9700 1 -- 172.21.15.12:0/214944575 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f831c089630 con 0x7f832c0b2f20 2023-07-08T12:38:37.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.390+0000 7f3e7d59a700 1 -- 172.21.15.12:0/3992339042 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74011830 con 0x7f3e800cb8a0 2023-07-08T12:38:37.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.390+0000 7f3e7d59a700 1 -- 172.21.15.12:0/3992339042 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e74011990 con 0x7f3e800cb8a0 2023-07-08T12:38:37.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.390+0000 7f3e7d59a700 1 -- 172.21.15.12:0/3992339042 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74011d20 con 0x7f3e800cb8a0 2023-07-08T12:38:37.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.391+0000 7f3e84d2c700 1 -- 172.21.15.12:0/3992339042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e800cb8a0 msgr2=0x7f3e8016c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.391+0000 7f3e84d2c700 1 --2- 172.21.15.12:0/3992339042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e800cb8a0 0x7f3e8016c570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3e800ca730 tx=0x7f3e74009b20 comp rx=0 tx=0).stop 2023-07-08T12:38:37.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.391+0000 7f832bfff700 1 --2- 172.21.15.12:0/214944575 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f831806ace0 0x7f831806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.391+0000 7f3cdb83b700 1 -- 172.21.15.12:0/347333965 wait complete. 2023-07-08T12:38:37.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.391+0000 7f3e84d2c700 1 -- 172.21.15.12:0/3992339042 shutdown_connections 2023-07-08T12:38:37.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.391+0000 7f3e84d2c700 1 --2- 172.21.15.12:0/3992339042 >> v2:172.21.15.12:3300/0 conn(0x7f3e80089570 0x7f3e800ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.391+0000 7f3e84d2c700 1 --2- 172.21.15.12:0/3992339042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e800cb8a0 0x7f3e8016c570 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.391+0000 7f3e84d2c700 1 -- 172.21.15.12:0/3992339042 >> 172.21.15.12:0/3992339042 conn(0x7f3e8006abc0 msgr2=0x7f3e800b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.391+0000 7f6586301700 1 -- 172.21.15.12:0/4173348686 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65801fa480 con 0x7f65800c9d40 2023-07-08T12:38:37.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.391+0000 7f3e84d2c700 1 -- 172.21.15.12:0/3992339042 shutdown_connections 2023-07-08T12:38:37.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.391+0000 7f657cff9700 1 -- 172.21.15.12:0/4173348686 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f657001a400 con 0x7f65800c9d40 2023-07-08T12:38:37.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.392+0000 7f657cff9700 1 --2- 172.21.15.12:0/4173348686 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f656c06adb0 0x7f656c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.392+0000 7f657cff9700 1 -- 172.21.15.12:0/4173348686 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6570089740 con 0x7f65800c9d40 2023-07-08T12:38:37.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.391+0000 7f3e84d2c700 1 -- 172.21.15.12:0/3992339042 wait complete. 2023-07-08T12:38:37.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.393+0000 7f91f8303700 1 -- 172.21.15.12:0/2473247703 shutdown_connections 2023-07-08T12:38:37.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.393+0000 7f91f8303700 1 --2- 172.21.15.12:0/2473247703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f0160700 0x7f91f0069f30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.393+0000 7f91f8303700 1 --2- 172.21.15.12:0/2473247703 >> v2:172.21.15.12:3300/0 conn(0x7f91f00c27b0 0x7f91f006a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.393+0000 7f91f8303700 1 -- 172.21.15.12:0/2473247703 >> 172.21.15.12:0/2473247703 conn(0x7f91f00cc290 msgr2=0x7f91f00cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.393+0000 7f3cdb83b700 1 Processor -- start 2023-07-08T12:38:37.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.393+0000 7f657ffff700 1 --2- 172.21.15.12:0/4173348686 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f656c06adb0 0x7f656c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.393+0000 7f3e84d2c700 1 Processor -- start 2023-07-08T12:38:37.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.393+0000 7f91f8303700 1 -- 172.21.15.12:0/2473247703 shutdown_connections 2023-07-08T12:38:37.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.393+0000 7f91f8303700 1 -- 172.21.15.12:0/2473247703 wait complete. 2023-07-08T12:38:37.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f91f8303700 1 Processor -- start 2023-07-08T12:38:37.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.394+0000 7f657ffff700 1 --2- 172.21.15.12:0/4173348686 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f656c06adb0 0x7f656c06d260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f657400d370 tx=0x7f6574012000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.393+0000 7f3cdb83b700 1 -- start start 2023-07-08T12:38:37.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.394+0000 7f3cdb83b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd40c9970 0x7f3cd41f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.394+0000 7f3cdb83b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd40c9d40 0x7f3cd4157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.394+0000 7f3cdb83b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd40cb6e0 0x7f3cd40c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.394+0000 7f3cdb83b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd4158070 con 0x7f3cd40c9970 2023-07-08T12:38:37.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.394+0000 7f3cdb83b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd4158310 con 0x7f3cd40c9d40 2023-07-08T12:38:37.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.394+0000 7f3cdb83b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41585f0 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.394+0000 7f3cd95d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd40c9970 0x7f3cd41f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.394+0000 7f3cd95d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd40c9970 0x7f3cd41f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49944/0 (socket says 172.21.15.12:49944) 2023-07-08T12:38:37.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.394+0000 7f3cd95d7700 1 -- 172.21.15.12:0/1709099881 learned_addr learned my addr 172.21.15.12:0/1709099881 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:37.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3cd9dd8700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd40cb6e0 0x7f3cd40c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3cd8dd6700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd40c9d40 0x7f3cd4157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.393+0000 7f3e84d2c700 1 -- start start 2023-07-08T12:38:37.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e84d2c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e80089570 0x7f3e800b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e84d2c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e800cb8a0 0x7f3e800c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e84d2c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cda10 0x7f3e800c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e84d2c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e800b4620 con 0x7f3e800cda10 2023-07-08T12:38:37.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e84d2c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e800c4100 con 0x7f3e80089570 2023-07-08T12:38:37.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e84d2c700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e800c43e0 con 0x7f3e800cb8a0 2023-07-08T12:38:37.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e7ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cda10 0x7f3e800c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e7ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cda10 0x7f3e800c13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49948/0 (socket says 172.21.15.12:49948) 2023-07-08T12:38:37.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e7ed9d700 1 -- 172.21.15.12:0/1735468421 learned_addr learned my addr 172.21.15.12:0/1735468421 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:37.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3cd9dd8700 1 -- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd40c9d40 msgr2=0x7f3cd4157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3cd9dd8700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd40c9d40 0x7f3cd4157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3cd9dd8700 1 -- 172.21.15.12:0/1709099881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd40c9970 msgr2=0x7f3cd41f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3cd9dd8700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd40c9970 0x7f3cd41f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3cd9dd8700 1 -- 172.21.15.12:0/1709099881 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc4009de0 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e7ed9d700 1 -- 172.21.15.12:0/1735468421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e800cb8a0 msgr2=0x7f3e800c0e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-08T12:38:37.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e7ed9d700 1 --2- 172.21.15.12:0/1735468421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e800cb8a0 0x7f3e800c0e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e7ed9d700 1 -- 172.21.15.12:0/1735468421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e80089570 msgr2=0x7f3e800b4040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-08T12:38:37.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e7ed9d700 1 --2- 172.21.15.12:0/1735468421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e80089570 0x7f3e800b4040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f3e7ed9d700 1 -- 172.21.15.12:0/1735468421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e740094a0 con 0x7f3e800cda10 2023-07-08T12:38:37.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f3cd9dd8700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd40cb6e0 0x7f3cd40c4810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00be00 tx=0x7f3ccc009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f3e7ed9d700 1 --2- 172.21.15.12:0/1735468421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cda10 0x7f3e800c13c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800b970 tx=0x7f3e7800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.395+0000 7f91f8303700 1 -- start start 2023-07-08T12:38:37.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f8303700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91f00c27b0 0x7f91f01636c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f8303700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f00ccda0 0x7f91f00b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f8303700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91f0160700 0x7f91f00b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f8303700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f0161600 con 0x7f91f0160700 2023-07-08T12:38:37.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f8303700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f0163c00 con 0x7f91f00ccda0 2023-07-08T12:38:37.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f8303700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f01f4be0 con 0x7f91f00c27b0 2023-07-08T12:38:37.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f609f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91f00c27b0 0x7f91f01636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f609f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91f00c27b0 0x7f91f01636c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59568/0 (socket says 172.21.15.12:59568) 2023-07-08T12:38:37.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f609f700 1 -- 172.21.15.12:0/2061173958 learned_addr learned my addr 172.21.15.12:0/2061173958 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:37.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f589e700 1 --2- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f00ccda0 0x7f91f00b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f609f700 1 -- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f00ccda0 msgr2=0x7f91f00b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f609f700 1 --2- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f00ccda0 0x7f91f00b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f609f700 1 -- 172.21.15.12:0/2061173958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91f0160700 msgr2=0x7f91f00b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f609f700 1 --2- 172.21.15.12:0/2061173958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91f0160700 0x7f91f00b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.396+0000 7f91f609f700 1 -- 172.21.15.12:0/2061173958 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00094a0 con 0x7f91f00c27b0 2023-07-08T12:38:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.397+0000 7f91f609f700 1 --2- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91f00c27b0 0x7f91f01636c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f91e00090b0 tx=0x7f91e0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.397+0000 7f8328ff9700 1 -- 172.21.15.12:0/214944575 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f832c0b2c40 con 0x7f832c0b2f20 2023-07-08T12:38:37.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.397+0000 7f832bfff700 1 --2- 172.21.15.12:0/214944575 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f831806ace0 0x7f831806d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f832000c2f0 tx=0x7f832000a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.397+0000 7f91e6ffd700 1 -- 172.21.15.12:0/2061173958 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e00038a0 con 0x7f91f00c27b0 2023-07-08T12:38:37.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.397+0000 7f91e6ffd700 1 -- 172.21.15.12:0/2061173958 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e001f070 con 0x7f91f00c27b0 2023-07-08T12:38:37.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.397+0000 7f91e6ffd700 1 -- 172.21.15.12:0/2061173958 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0003c60 con 0x7f91f00c27b0 2023-07-08T12:38:37.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.397+0000 7f91f8303700 1 -- 172.21.15.12:0/2061173958 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f0160c60 con 0x7f91f00c27b0 2023-07-08T12:38:37.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.397+0000 7f91f8303700 1 -- 172.21.15.12:0/2061173958 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f0161220 con 0x7f91f00c27b0 2023-07-08T12:38:37.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.398+0000 7f3cc9ffb700 1 -- 172.21.15.12:0/1709099881 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc00cc10 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.398+0000 7f3cc9ffb700 1 -- 172.21.15.12:0/1709099881 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc00cd70 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.398+0000 7f3cc9ffb700 1 -- 172.21.15.12:0/1709099881 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc003c40 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.398+0000 7f3cdb83b700 1 -- 172.21.15.12:0/1709099881 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd41588d0 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.398+0000 7f3cdb83b700 1 -- 172.21.15.12:0/1709099881 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd415a5c0 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.400+0000 7f3cdb83b700 1 -- 172.21.15.12:0/1709099881 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd40b1690 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.400+0000 7f91e6ffd700 1 -- 172.21.15.12:0/2061173958 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f91e001a430 con 0x7f91f00c27b0 2023-07-08T12:38:37.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.400+0000 7f91e6ffd700 1 --2- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f91d406ace0 0x7f91d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.400+0000 7f91e6ffd700 1 -- 172.21.15.12:0/2061173958 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f91e0021070 con 0x7f91f00c27b0 2023-07-08T12:38:37.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.401+0000 7f3e6effd700 1 -- 172.21.15.12:0/1735468421 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7800c9b0 con 0x7f3e800cda10 2023-07-08T12:38:37.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.401+0000 7f657cff9700 1 -- 172.21.15.12:0/4173348686 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6570054570 con 0x7f65800c9d40 2023-07-08T12:38:37.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.401+0000 7f3e6effd700 1 -- 172.21.15.12:0/1735468421 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7800cb10 con 0x7f3e800cda10 2023-07-08T12:38:37.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.401+0000 7f3e6effd700 1 -- 172.21.15.12:0/1735468421 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78014440 con 0x7f3e800cda10 2023-07-08T12:38:37.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.401+0000 7f3e84d2c700 1 -- 172.21.15.12:0/1735468421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e800c46c0 con 0x7f3e800cda10 2023-07-08T12:38:37.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.401+0000 7f3e84d2c700 1 -- 172.21.15.12:0/1735468421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e800c4a30 con 0x7f3e800cda10 2023-07-08T12:38:37.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.401+0000 7f91f589e700 1 --2- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f91d406ace0 0x7f91d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.401+0000 7f3cc9ffb700 1 -- 172.21.15.12:0/1709099881 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ccc014440 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.402+0000 7f91f589e700 1 --2- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f91d406ace0 0x7f91d406d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f91f00cd390 tx=0x7f91ec009110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.402+0000 7f3cc9ffb700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3cb806ad30 0x7f3cb806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.402+0000 7f3cc9ffb700 1 -- 172.21.15.12:0/1709099881 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3ccc089a10 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.402+0000 7f3cd95d7700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3cb806ad30 0x7f3cb806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.403+0000 7f3cd95d7700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3cb806ad30 0x7f3cb806d1e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000ac10 tx=0x7f3cd0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.403+0000 7f3e6effd700 1 -- 172.21.15.12:0/1735468421 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e780145a0 con 0x7f3e800cda10 2023-07-08T12:38:37.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.403+0000 7f3e6effd700 1 --2- 172.21.15.12:0/1735468421 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e6006ace0 0x7f3e6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.403+0000 7f3e6effd700 1 -- 172.21.15.12:0/1735468421 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3e78006e40 con 0x7f3e800cda10 2023-07-08T12:38:37.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.403+0000 7f3e7e59c700 1 --2- 172.21.15.12:0/1735468421 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e6006ace0 0x7f3e6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.404+0000 7f3e7e59c700 1 --2- 172.21.15.12:0/1735468421 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e6006ace0 0x7f3e6006d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3e8016baa0 tx=0x7f3e700079f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.405+0000 7f3cc9ffb700 1 -- 172.21.15.12:0/1709099881 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3ccc054940 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.406+0000 7fd5856b3700 1 Processor -- start 2023-07-08T12:38:37.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.406+0000 7fd5856b3700 1 -- start start 2023-07-08T12:38:37.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.406+0000 7f3e84d2c700 1 -- 172.21.15.12:0/1735468421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e800b1690 con 0x7f3e800cda10 2023-07-08T12:38:37.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.407+0000 7fd5856b3700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd58015a190 0x7fd58015e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.407+0000 7fd5856b3700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd5800cd940 0x7fd5800cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.407+0000 7fd5856b3700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5800ce4b0 con 0x7fd5800cb130 2023-07-08T12:38:37.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.407+0000 7fd5856b3700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58015ec40 con 0x7fd5800cd940 2023-07-08T12:38:37.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.407+0000 7fd5856b3700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5800ccc70 con 0x7fd58015a190 2023-07-08T12:38:37.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.407+0000 7fd57f7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd5800cd940 0x7fd5800cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.407+0000 7fd57f7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd5800cd940 0x7fd5800cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49964/0 (socket says 172.21.15.12:49964) 2023-07-08T12:38:37.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.407+0000 7fd57f7fe700 1 -- 172.21.15.12:0/610309507 learned_addr learned my addr 172.21.15.12:0/610309507 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:37.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.408+0000 7fd57f7fe700 1 -- 172.21.15.12:0/610309507 >> v2:172.21.15.160:3300/0 conn(0x7fd58015a190 msgr2=0x7fd58015e700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-08T12:38:37.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.408+0000 7fd57f7fe700 1 --2- 172.21.15.12:0/610309507 >> v2:172.21.15.160:3300/0 conn(0x7fd58015a190 0x7fd58015e700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.408+0000 7fd57f7fe700 1 -- 172.21.15.12:0/610309507 >> v1:172.21.15.12:6789/0 conn(0x7fd5800cb130 legacy=0x7fd5800cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.408+0000 7fd57f7fe700 1 -- 172.21.15.12:0/610309507 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5800ccf50 con 0x7fd5800cd940 2023-07-08T12:38:37.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.408+0000 7fd57f7fe700 1 --2- 172.21.15.12:0/610309507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5800cd940 0x7fd5800cef00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd58006f940 tx=0x7fd57800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4632741c9c017457 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.411+0000 7f91f8303700 1 -- 172.21.15.12:0/2061173958 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d80052f0 con 0x7f91f00c27b0 2023-07-08T12:38:37.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.412+0000 7fd57dffb700 1 -- 172.21.15.12:0/610309507 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57800e070 con 0x7fd5800cd940 2023-07-08T12:38:37.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.412+0000 7fd57dffb700 1 -- 172.21.15.12:0/610309507 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57800b9c0 con 0x7fd5800cd940 2023-07-08T12:38:37.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.412+0000 7fd57dffb700 1 -- 172.21.15.12:0/610309507 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57800bd70 con 0x7fd5800cd940 2023-07-08T12:38:37.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.412+0000 7f3e6effd700 1 -- 172.21.15.12:0/1735468421 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3e800b1690 con 0x7f3e800cda10 2023-07-08T12:38:37.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.413+0000 7fd5856b3700 1 -- 172.21.15.12:0/610309507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5800cd940 msgr2=0x7fd5800cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.413+0000 7fd5856b3700 1 --2- 172.21.15.12:0/610309507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5800cd940 0x7fd5800cef00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd58006f940 tx=0x7fd57800b330 comp rx=0 tx=0).stop 2023-07-08T12:38:37.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.413+0000 7fd5856b3700 1 -- 172.21.15.12:0/610309507 shutdown_connections 2023-07-08T12:38:37.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.413+0000 7fd5856b3700 1 --2- 172.21.15.12:0/610309507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5800cd940 0x7fd5800cef00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.413+0000 7fd5856b3700 1 --2- 172.21.15.12:0/610309507 >> v2:172.21.15.160:3300/0 conn(0x7fd58015a190 0x7fd58015e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.413+0000 7fd5856b3700 1 -- 172.21.15.12:0/610309507 >> 172.21.15.12:0/610309507 conn(0x7fd58006e550 msgr2=0x7fd58006e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.414+0000 7fd5856b3700 1 -- 172.21.15.12:0/610309507 shutdown_connections 2023-07-08T12:38:37.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.414+0000 7fd5856b3700 1 -- 172.21.15.12:0/610309507 wait complete. 2023-07-08T12:38:37.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.415+0000 7fd5856b3700 1 Processor -- start 2023-07-08T12:38:37.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.415+0000 7fd5856b3700 1 -- start start 2023-07-08T12:38:37.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.416+0000 7f91e6ffd700 1 -- 172.21.15.12:0/2061173958 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f91e0055f30 con 0x7f91f00c27b0 2023-07-08T12:38:37.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.416+0000 7fd5856b3700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5800cb130 0x7fd5800af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.416+0000 7fd5856b3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5800cd940 0x7fd5800c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.416+0000 7fd5856b3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5800b17d0 0x7fd5800c4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.416+0000 7fd5856b3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5800b0d30 con 0x7fd5800b17d0 2023-07-08T12:38:37.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.416+0000 7fd5856b3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58006c070 con 0x7fd5800cd940 2023-07-08T12:38:37.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.416+0000 7fd5856b3700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58006c350 con 0x7fd5800cb130 2023-07-08T12:38:37.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.416+0000 7fd57effd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5800cb130 0x7fd5800af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.416+0000 7fd57effd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5800cb130 0x7fd5800af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59586/0 (socket says 172.21.15.12:59586) 2023-07-08T12:38:37.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.416+0000 7fd57effd700 1 -- 172.21.15.12:0/4177367824 learned_addr learned my addr 172.21.15.12:0/4177367824 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:37.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.417+0000 7fd57f7fe700 1 --2- 172.21.15.12:0/4177367824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5800b17d0 0x7fd5800c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.417+0000 7fd57f7fe700 1 -- 172.21.15.12:0/4177367824 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5800cb130 msgr2=0x7fd5800af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.417+0000 7fd57f7fe700 1 --2- 172.21.15.12:0/4177367824 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5800cb130 0x7fd5800af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.417+0000 7fd57f7fe700 1 -- 172.21.15.12:0/4177367824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5800cd940 msgr2=0x7fd5800c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.417+0000 7fd57f7fe700 1 --2- 172.21.15.12:0/4177367824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5800cd940 0x7fd5800c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.417+0000 7fd57f7fe700 1 -- 172.21.15.12:0/4177367824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd578009b30 con 0x7fd5800b17d0 2023-07-08T12:38:37.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.418+0000 7ff8791e9700 1 Processor -- start 2023-07-08T12:38:37.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.418+0000 7ff8791e9700 1 -- start start 2023-07-08T12:38:37.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.419+0000 7ff8791e9700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb8b0 0x7ff874157d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.419+0000 7ff8791e9700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8740ce5d0 con 0x7ff8740c9970 2023-07-08T12:38:37.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.419+0000 7ff8791e9700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8740cce20 con 0x7ff8740c9d40 2023-07-08T12:38:37.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.419+0000 7ff8791e9700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740cd100 con 0x7ff8740cb8b0 2023-07-08T12:38:37.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.418+0000 7fd57f7fe700 1 --2- 172.21.15.12:0/4177367824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5800b17d0 0x7fd5800c4550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd578015000 tx=0x7fd578004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.418+0000 7fd5677fe700 1 -- 172.21.15.12:0/4177367824 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578004930 con 0x7fd5800b17d0 2023-07-08T12:38:37.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.418+0000 7fd5677fe700 1 -- 172.21.15.12:0/4177367824 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5780099b0 con 0x7fd5800b17d0 2023-07-08T12:38:37.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.418+0000 7fd5856b3700 1 -- 172.21.15.12:0/4177367824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5800b0fd0 con 0x7fd5800b17d0 2023-07-08T12:38:37.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.418+0000 7fd5856b3700 1 -- 172.21.15.12:0/4177367824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58006bcb0 con 0x7fd5800b17d0 2023-07-08T12:38:37.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.418+0000 7fd5677fe700 1 -- 172.21.15.12:0/4177367824 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57801c660 con 0x7fd5800b17d0 2023-07-08T12:38:37.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.419+0000 7ff87359e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb8b0 0x7ff874157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.420+0000 7ff87359e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb8b0 0x7ff874157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59592/0 (socket says 172.21.15.12:59592) 2023-07-08T12:38:37.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.420+0000 7ff87359e700 1 -- 172.21.15.12:0/360598998 learned_addr learned my addr 172.21.15.12:0/360598998 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:37.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.420+0000 7fd5677fe700 1 -- 172.21.15.12:0/4177367824 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd57801c7c0 con 0x7fd5800b17d0 2023-07-08T12:38:37.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.421+0000 7fd5677fe700 1 --2- 172.21.15.12:0/4177367824 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd56c06ace0 0x7fd56c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.421+0000 7fd5677fe700 1 -- 172.21.15.12:0/4177367824 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd57808b160 con 0x7fd5800b17d0 2023-07-08T12:38:37.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.422+0000 7fd57effd700 1 --2- 172.21.15.12:0/4177367824 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd56c06ace0 0x7fd56c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.422+0000 7fd57effd700 1 --2- 172.21.15.12:0/4177367824 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd56c06ace0 0x7fd56c06d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd570009a30 tx=0x7fd570009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.425+0000 7fd5856b3700 1 -- 172.21.15.12:0/4177367824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5600052f0 con 0x7fd5800b17d0 2023-07-08T12:38:37.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.425+0000 7ff87359e700 1 -- 172.21.15.12:0/360598998 >> v1:172.21.15.160:6789/0 conn(0x7ff8740c9d40 legacy=0x7ff8740cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.425+0000 7ff87359e700 1 -- 172.21.15.12:0/360598998 >> v1:172.21.15.12:6789/0 conn(0x7ff8740c9970 legacy=0x7ff8740cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.425+0000 7ff87359e700 1 -- 172.21.15.12:0/360598998 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8740cd3e0 con 0x7ff8740cb8b0 2023-07-08T12:38:37.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.426+0000 7ff87359e700 1 --2- 172.21.15.12:0/360598998 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb8b0 0x7ff874157d60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff868008950 tx=0x7ff86800d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=728d33ebde6483f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.434+0000 7fd5677fe700 1 -- 172.21.15.12:0/4177367824 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd578055f10 con 0x7fd5800b17d0 2023-07-08T12:38:37.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.434+0000 7ff871d9b700 1 -- 172.21.15.12:0/360598998 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868010070 con 0x7ff8740cb8b0 2023-07-08T12:38:37.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.434+0000 7ff871d9b700 1 -- 172.21.15.12:0/360598998 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86800d980 con 0x7ff8740cb8b0 2023-07-08T12:38:37.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.434+0000 7ff871d9b700 1 -- 172.21.15.12:0/360598998 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86800dd10 con 0x7ff8740cb8b0 2023-07-08T12:38:37.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.435+0000 7ff8791e9700 1 -- 172.21.15.12:0/360598998 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb8b0 msgr2=0x7ff874157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.435+0000 7ff8791e9700 1 --2- 172.21.15.12:0/360598998 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb8b0 0x7ff874157d60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff868008950 tx=0x7ff86800d330 comp rx=0 tx=0).stop 2023-07-08T12:38:37.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.436+0000 7ff8791e9700 1 -- 172.21.15.12:0/360598998 shutdown_connections 2023-07-08T12:38:37.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.436+0000 7ff8791e9700 1 --2- 172.21.15.12:0/360598998 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb8b0 0x7ff874157d60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.436+0000 7ff8791e9700 1 -- 172.21.15.12:0/360598998 >> 172.21.15.12:0/360598998 conn(0x7ff87406abc0 msgr2=0x7ff8740b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.436+0000 7ff8791e9700 1 -- 172.21.15.12:0/360598998 shutdown_connections 2023-07-08T12:38:37.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.436+0000 7ff8791e9700 1 -- 172.21.15.12:0/360598998 wait complete. 2023-07-08T12:38:37.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.438+0000 7ff8791e9700 1 Processor -- start 2023-07-08T12:38:37.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.438+0000 7ff8791e9700 1 -- start start 2023-07-08T12:38:37.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.440+0000 7ff8791e9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8740c9970 0x7ff8740b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.440+0000 7ff8791e9700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740c9d40 0x7ff8741f1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.440+0000 7ff8791e9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8740cb8b0 0x7ff8741feba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.440+0000 7ff8791e9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8741f20c0 con 0x7ff8740c9970 2023-07-08T12:38:37.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.440+0000 7ff8791e9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff874159260 con 0x7ff8740cb8b0 2023-07-08T12:38:37.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.440+0000 7ff8791e9700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8741586e0 con 0x7ff8740c9d40 2023-07-08T12:38:37.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.440+0000 7ff872d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8740c9970 0x7ff8740b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.440+0000 7ff872d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8740c9970 0x7ff8740b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49982/0 (socket says 172.21.15.12:49982) 2023-07-08T12:38:37.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.440+0000 7ff872d9d700 1 -- 172.21.15.12:0/3817708145 learned_addr learned my addr 172.21.15.12:0/3817708145 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:37.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.447+0000 7ff872d9d700 1 -- 172.21.15.12:0/3817708145 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740c9d40 msgr2=0x7ff8741f1120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-08T12:38:37.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.447+0000 7ff872d9d700 1 --2- 172.21.15.12:0/3817708145 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740c9d40 0x7ff8741f1120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.447+0000 7ff872d9d700 1 -- 172.21.15.12:0/3817708145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8740cb8b0 msgr2=0x7ff8741feba0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-08T12:38:37.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.447+0000 7ff872d9d700 1 --2- 172.21.15.12:0/3817708145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8740cb8b0 0x7ff8741feba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.447+0000 7ff872d9d700 1 -- 172.21.15.12:0/3817708145 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8680085d0 con 0x7ff8740c9970 2023-07-08T12:38:37.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.447+0000 7ff872d9d700 1 --2- 172.21.15.12:0/3817708145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8740c9970 0x7ff8740b04e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00cbc0 tx=0x7ff86c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.448+0000 7ff85b7fe700 1 -- 172.21.15.12:0/3817708145 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c003c00 con 0x7ff8740c9970 2023-07-08T12:38:37.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.448+0000 7ff85b7fe700 1 -- 172.21.15.12:0/3817708145 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c003d60 con 0x7ff8740c9970 2023-07-08T12:38:37.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.448+0000 7ff85b7fe700 1 -- 172.21.15.12:0/3817708145 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c00abb0 con 0x7ff8740c9970 2023-07-08T12:38:37.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.448+0000 7ff8791e9700 1 -- 172.21.15.12:0/3817708145 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8741589c0 con 0x7ff8740c9970 2023-07-08T12:38:37.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.448+0000 7ff8791e9700 1 -- 172.21.15.12:0/3817708145 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff874159470 con 0x7ff8740c9970 2023-07-08T12:38:37.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.452+0000 7ff85b7fe700 1 -- 172.21.15.12:0/3817708145 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff86c00ad10 con 0x7ff8740c9970 2023-07-08T12:38:37.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.453+0000 7ff85b7fe700 1 --2- 172.21.15.12:0/3817708145 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff86006adb0 0x7ff86006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:37.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.453+0000 7ff85b7fe700 1 -- 172.21.15.12:0/3817708145 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff86c089780 con 0x7ff8740c9970 2023-07-08T12:38:37.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.454+0000 7ff87259c700 1 --2- 172.21.15.12:0/3817708145 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff86006adb0 0x7ff86006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:37.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.454+0000 7ff87259c700 1 --2- 172.21.15.12:0/3817708145 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff86006adb0 0x7ff86006d260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff8740b0a20 tx=0x7ff864007a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:37.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.457+0000 7ff8791e9700 1 -- 172.21.15.12:0/3817708145 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8540052f0 con 0x7ff8740c9970 2023-07-08T12:38:37.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.470+0000 7ff85b7fe700 1 -- 172.21.15.12:0/3817708145 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff86c0545b0 con 0x7ff8740c9970 2023-07-08T12:38:37.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.743+0000 7fd930222700 1 -- 172.21.15.12:0/1302984672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd9100055a0 con 0x7fd9280c5fe0 2023-07-08T12:38:37.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.744+0000 7fd91e7fc700 1 -- 172.21.15.12:0/1302984672 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd920017070 con 0x7fd9280c5fe0 2023-07-08T12:38:37.748 INFO:teuthology.orchestra.run.smithi012.stdout:55834574859 2023-07-08T12:38:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.748+0000 7fd930222700 1 -- 172.21.15.12:0/1302984672 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd90c06ad80 msgr2=0x7fd90c06d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.748+0000 7fd930222700 1 --2- 172.21.15.12:0/1302984672 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd90c06ad80 0x7fd90c06d230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd9280cc560 tx=0x7fd924006c60 comp rx=0 tx=0).stop 2023-07-08T12:38:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.748+0000 7fd930222700 1 -- 172.21.15.12:0/1302984672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9280c5fe0 msgr2=0x7fd9280c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.748+0000 7fd930222700 1 --2- 172.21.15.12:0/1302984672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9280c5fe0 0x7fd9280c6430 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd9280cbc80 tx=0x7fd92000ce00 comp rx=0 tx=0).stop 2023-07-08T12:38:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.748+0000 7fd930222700 1 -- 172.21.15.12:0/1302984672 shutdown_connections 2023-07-08T12:38:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.748+0000 7fd930222700 1 --2- 172.21.15.12:0/1302984672 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd90c06ad80 0x7fd90c06d230 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd9280cc560 tx=0x7fd924006c60 comp rx=0 tx=0).stop 2023-07-08T12:38:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.748+0000 7fd930222700 1 --2- 172.21.15.12:0/1302984672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9280c5fe0 0x7fd9280c6430 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.748+0000 7fd930222700 1 --2- 172.21.15.12:0/1302984672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd914004850 0x7fd9280c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.748+0000 7fd930222700 1 --2- 172.21.15.12:0/1302984672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9281f1ed0 0x7fd9280c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.748+0000 7fd930222700 1 -- 172.21.15.12:0/1302984672 >> 172.21.15.12:0/1302984672 conn(0x7fd92806abc0 msgr2=0x7fd9280ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.748+0000 7fd930222700 1 -- 172.21.15.12:0/1302984672 shutdown_connections 2023-07-08T12:38:37.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.748+0000 7fd930222700 1 -- 172.21.15.12:0/1302984672 wait complete. 2023-07-08T12:38:37.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2023-07-08T12:38:37.769 DEBUG:teuthology.parallel:result is None 2023-07-08T12:38:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.808+0000 7f91f8303700 1 -- 172.21.15.12:0/2061173958 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f91d80055a0 con 0x7f91f00c27b0 2023-07-08T12:38:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.809+0000 7f91e6ffd700 1 -- 172.21.15.12:0/2061173958 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f91e002b020 con 0x7f91f00c27b0 2023-07-08T12:38:37.811 INFO:teuthology.orchestra.run.smithi012.stdout:55834574860 2023-07-08T12:38:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.811+0000 7fd5856b3700 1 -- 172.21.15.12:0/4177367824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd5600055a0 con 0x7fd5800b17d0 2023-07-08T12:38:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.811+0000 7fd5677fe700 1 -- 172.21.15.12:0/4177367824 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd578059530 con 0x7fd5800b17d0 2023-07-08T12:38:37.813 INFO:teuthology.orchestra.run.smithi012.stdout:55834574860 2023-07-08T12:38:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.813+0000 7f91f8303700 1 -- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f91d406ace0 msgr2=0x7f91d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.813+0000 7f91f8303700 1 --2- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f91d406ace0 0x7f91d406d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f91f00cd390 tx=0x7f91ec009110 comp rx=0 tx=0).stop 2023-07-08T12:38:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.813+0000 7f91f8303700 1 -- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91f00c27b0 msgr2=0x7f91f01636c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.813+0000 7f91f8303700 1 --2- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91f00c27b0 0x7f91f01636c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f91e00090b0 tx=0x7f91e0004750 comp rx=0 tx=0).stop 2023-07-08T12:38:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7f91f8303700 1 -- 172.21.15.12:0/2061173958 shutdown_connections 2023-07-08T12:38:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7f91f8303700 1 --2- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f91d406ace0 0x7f91d406d190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7f91f8303700 1 --2- 172.21.15.12:0/2061173958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91f0160700 0x7f91f00b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7f91f8303700 1 --2- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91f00c27b0 0x7f91f01636c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7f91f8303700 1 --2- 172.21.15.12:0/2061173958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f00ccda0 0x7f91f00b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7f91f8303700 1 -- 172.21.15.12:0/2061173958 >> 172.21.15.12:0/2061173958 conn(0x7f91f00cc290 msgr2=0x7f91f0071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7fd5856b3700 1 -- 172.21.15.12:0/4177367824 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd56c06ace0 msgr2=0x7fd56c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7fd5856b3700 1 --2- 172.21.15.12:0/4177367824 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd56c06ace0 0x7fd56c06d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd570009a30 tx=0x7fd570009190 comp rx=0 tx=0).stop 2023-07-08T12:38:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7fd5856b3700 1 -- 172.21.15.12:0/4177367824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5800b17d0 msgr2=0x7fd5800c4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7fd5856b3700 1 --2- 172.21.15.12:0/4177367824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5800b17d0 0x7fd5800c4550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd578015000 tx=0x7fd578004ad0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7fd5856b3700 1 -- 172.21.15.12:0/4177367824 shutdown_connections 2023-07-08T12:38:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7fd5856b3700 1 --2- 172.21.15.12:0/4177367824 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd56c06ace0 0x7fd56c06d190 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd570009a30 tx=0x7fd570009190 comp rx=0 tx=0).stop 2023-07-08T12:38:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7fd5856b3700 1 --2- 172.21.15.12:0/4177367824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5800b17d0 0x7fd5800c4550 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7fd5856b3700 1 --2- 172.21.15.12:0/4177367824 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5800cb130 0x7fd5800af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7fd5856b3700 1 --2- 172.21.15.12:0/4177367824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5800cd940 0x7fd5800c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.814+0000 7fd5856b3700 1 -- 172.21.15.12:0/4177367824 >> 172.21.15.12:0/4177367824 conn(0x7fd58006e550 msgr2=0x7fd5800d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.815+0000 7f8332a41700 1 -- 172.21.15.12:0/214944575 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f832c0b2c40 con 0x7f832c0b2f20 2023-07-08T12:38:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.815+0000 7fd5856b3700 1 -- 172.21.15.12:0/4177367824 shutdown_connections 2023-07-08T12:38:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.815+0000 7fd5856b3700 1 -- 172.21.15.12:0/4177367824 wait complete. 2023-07-08T12:38:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.815+0000 7f8328ff9700 1 -- 172.21.15.12:0/214944575 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f832c0b2c40 con 0x7f832c0b2f20 2023-07-08T12:38:37.822 INFO:teuthology.orchestra.run.smithi012.stdout:55834574861 2023-07-08T12:38:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.819+0000 7f91f8303700 1 -- 172.21.15.12:0/2061173958 shutdown_connections 2023-07-08T12:38:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.819+0000 7f8332a41700 1 -- 172.21.15.12:0/214944575 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f831806ace0 msgr2=0x7f831806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.819+0000 7f8332a41700 1 --2- 172.21.15.12:0/214944575 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f831806ace0 0x7f831806d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f832000c2f0 tx=0x7f832000a000 comp rx=0 tx=0).stop 2023-07-08T12:38:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.819+0000 7f8332a41700 1 -- 172.21.15.12:0/214944575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c0b2f20 msgr2=0x7f832c0b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.819+0000 7f8332a41700 1 --2- 172.21.15.12:0/214944575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c0b2f20 0x7f832c0b3390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f831c00e960 tx=0x7f831c00ec70 comp rx=0 tx=0).stop 2023-07-08T12:38:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.820+0000 7f6586301700 1 -- 172.21.15.12:0/4173348686 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f65800ce380 con 0x7f65800c9d40 2023-07-08T12:38:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.820+0000 7f657cff9700 1 -- 172.21.15.12:0/4173348686 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6570015070 con 0x7f65800c9d40 2023-07-08T12:38:37.825 INFO:teuthology.orchestra.run.smithi012.stdout:55834574859 2023-07-08T12:38:37.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.823+0000 7f8332a41700 1 -- 172.21.15.12:0/214944575 shutdown_connections 2023-07-08T12:38:37.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.823+0000 7f8332a41700 1 --2- 172.21.15.12:0/214944575 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f831806ace0 0x7f831806d190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.823+0000 7f8332a41700 1 --2- 172.21.15.12:0/214944575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f832c0c2700 0x7f832c0c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.823+0000 7f8332a41700 1 --2- 172.21.15.12:0/214944575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8314003a00 0x7f832c14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.823+0000 7f8332a41700 1 --2- 172.21.15.12:0/214944575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c0b2f20 0x7f832c0b3390 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.823+0000 7f8332a41700 1 -- 172.21.15.12:0/214944575 >> 172.21.15.12:0/214944575 conn(0x7f832c0cb9f0 msgr2=0x7f832c0af920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.823+0000 7f8332a41700 1 -- 172.21.15.12:0/214944575 shutdown_connections 2023-07-08T12:38:37.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.823+0000 7f8332a41700 1 -- 172.21.15.12:0/214944575 wait complete. 2023-07-08T12:38:37.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.823+0000 7f6586301700 1 -- 172.21.15.12:0/4173348686 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f656c06adb0 msgr2=0x7f656c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.823+0000 7f6586301700 1 --2- 172.21.15.12:0/4173348686 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f656c06adb0 0x7f656c06d260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f657400d370 tx=0x7f6574012000 comp rx=0 tx=0).stop 2023-07-08T12:38:37.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.823+0000 7f6586301700 1 -- 172.21.15.12:0/4173348686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65800c9d40 msgr2=0x7f65801fa780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.823+0000 7f6586301700 1 --2- 172.21.15.12:0/4173348686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65800c9d40 0x7f65801fa780 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f657000b550 tx=0x7f657000b860 comp rx=0 tx=0).stop 2023-07-08T12:38:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.824+0000 7f91f8303700 1 -- 172.21.15.12:0/2061173958 wait complete. 2023-07-08T12:38:37.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.824+0000 7f6586301700 1 -- 172.21.15.12:0/4173348686 shutdown_connections 2023-07-08T12:38:37.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.824+0000 7f6586301700 1 --2- 172.21.15.12:0/4173348686 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f656c06adb0 0x7f656c06d260 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.824+0000 7f6586301700 1 --2- 172.21.15.12:0/4173348686 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65800cb6e0 0x7f65801f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.824+0000 7f6586301700 1 --2- 172.21.15.12:0/4173348686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65800c9970 0x7f65800c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.824+0000 7f6586301700 1 --2- 172.21.15.12:0/4173348686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65800c9d40 0x7f65801fa780 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.824+0000 7f6586301700 1 -- 172.21.15.12:0/4173348686 >> 172.21.15.12:0/4173348686 conn(0x7f658006abc0 msgr2=0x7f6580156f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.824+0000 7f6586301700 1 -- 172.21.15.12:0/4173348686 shutdown_connections 2023-07-08T12:38:37.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.824+0000 7f6586301700 1 -- 172.21.15.12:0/4173348686 wait complete. 2023-07-08T12:38:37.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.824+0000 7f3cdb83b700 1 -- 172.21.15.12:0/1709099881 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3cd40b1a70 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.825+0000 7f3cc9ffb700 1 -- 172.21.15.12:0/1709099881 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3cd40b1a70 con 0x7f3cd40cb6e0 2023-07-08T12:38:37.835 INFO:teuthology.orchestra.run.smithi012.stdout:55834574860 2023-07-08T12:38:37.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.825+0000 7f3e84d2c700 1 -- 172.21.15.12:0/1735468421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3e800b1690 con 0x7f3e800cda10 2023-07-08T12:38:37.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.825+0000 7f3e6effd700 1 -- 172.21.15.12:0/1735468421 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e800b1690 con 0x7f3e800cda10 2023-07-08T12:38:37.837 INFO:teuthology.orchestra.run.smithi012.stdout:55834574860 2023-07-08T12:38:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.828+0000 7f3e84d2c700 1 -- 172.21.15.12:0/1735468421 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e6006ace0 msgr2=0x7f3e6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.828+0000 7f3e84d2c700 1 --2- 172.21.15.12:0/1735468421 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e6006ace0 0x7f3e6006d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3e8016baa0 tx=0x7f3e700079f0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.828+0000 7f3e84d2c700 1 -- 172.21.15.12:0/1735468421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cda10 msgr2=0x7f3e800c13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.828+0000 7f3e84d2c700 1 --2- 172.21.15.12:0/1735468421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cda10 0x7f3e800c13c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800b970 tx=0x7f3e7800bd30 comp rx=0 tx=0).stop 2023-07-08T12:38:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.829+0000 7f3cdb83b700 1 -- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3cb806ad30 msgr2=0x7f3cb806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.829+0000 7f3cdb83b700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3cb806ad30 0x7f3cb806d1e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000ac10 tx=0x7f3cd0008040 comp rx=0 tx=0).stop 2023-07-08T12:38:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.829+0000 7f3cdb83b700 1 -- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd40cb6e0 msgr2=0x7f3cd40c4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.829+0000 7f3cdb83b700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd40cb6e0 0x7f3cd40c4810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00be00 tx=0x7f3ccc009ed0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.829+0000 7f3cdb83b700 1 -- 172.21.15.12:0/1709099881 shutdown_connections 2023-07-08T12:38:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.829+0000 7f3cdb83b700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3cb806ad30 0x7f3cb806d1e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.829+0000 7f3cdb83b700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd40cb6e0 0x7f3cd40c4810 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.829+0000 7f3cdb83b700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd40c9970 0x7f3cd41f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.829+0000 7f3cdb83b700 1 --2- 172.21.15.12:0/1709099881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd40c9d40 0x7f3cd4157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.829+0000 7f3cdb83b700 1 -- 172.21.15.12:0/1709099881 >> 172.21.15.12:0/1709099881 conn(0x7f3cd406abc0 msgr2=0x7f3cd4151640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.829+0000 7f3cdb83b700 1 -- 172.21.15.12:0/1709099881 shutdown_connections 2023-07-08T12:38:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.829+0000 7f3cdb83b700 1 -- 172.21.15.12:0/1709099881 wait complete. 2023-07-08T12:38:37.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.4 2023-07-08T12:38:37.842 DEBUG:teuthology.parallel:result is None 2023-07-08T12:38:37.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.835+0000 7f3e84d2c700 1 -- 172.21.15.12:0/1735468421 shutdown_connections 2023-07-08T12:38:37.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.835+0000 7f3e84d2c700 1 --2- 172.21.15.12:0/1735468421 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e6006ace0 0x7f3e6006d190 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.835+0000 7f3e84d2c700 1 --2- 172.21.15.12:0/1735468421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e800cda10 0x7f3e800c13c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.835+0000 7f3e84d2c700 1 --2- 172.21.15.12:0/1735468421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e80089570 0x7f3e800b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.835+0000 7f3e84d2c700 1 --2- 172.21.15.12:0/1735468421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e800cb8a0 0x7f3e800c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.835+0000 7f3e84d2c700 1 -- 172.21.15.12:0/1735468421 >> 172.21.15.12:0/1735468421 conn(0x7f3e8006abc0 msgr2=0x7f3e800ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.835+0000 7f3e84d2c700 1 -- 172.21.15.12:0/1735468421 shutdown_connections 2023-07-08T12:38:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.835+0000 7f3e84d2c700 1 -- 172.21.15.12:0/1735468421 wait complete. 2023-07-08T12:38:37.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574861 for osd.7 2023-07-08T12:38:37.844 DEBUG:teuthology.parallel:result is None 2023-07-08T12:38:37.849 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2023-07-08T12:38:37.849 DEBUG:teuthology.parallel:result is None 2023-07-08T12:38:37.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.2 2023-07-08T12:38:37.852 DEBUG:teuthology.parallel:result is None 2023-07-08T12:38:37.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.3 2023-07-08T12:38:37.857 DEBUG:teuthology.parallel:result is None 2023-07-08T12:38:37.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.6 2023-07-08T12:38:37.858 DEBUG:teuthology.parallel:result is None 2023-07-08T12:38:37.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.862+0000 7ff8791e9700 1 -- 172.21.15.12:0/3817708145 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff8540055a0 con 0x7ff8740c9970 2023-07-08T12:38:37.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.863+0000 7ff85b7fe700 1 -- 172.21.15.12:0/3817708145 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff86c057bd0 con 0x7ff8740c9970 2023-07-08T12:38:37.864 INFO:teuthology.orchestra.run.smithi012.stdout:55834574860 2023-07-08T12:38:37.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.865+0000 7ff8791e9700 1 -- 172.21.15.12:0/3817708145 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff86006adb0 msgr2=0x7ff86006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.865+0000 7ff8791e9700 1 --2- 172.21.15.12:0/3817708145 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff86006adb0 0x7ff86006d260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff8740b0a20 tx=0x7ff864007a40 comp rx=0 tx=0).stop 2023-07-08T12:38:37.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.865+0000 7ff8791e9700 1 -- 172.21.15.12:0/3817708145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8740c9970 msgr2=0x7ff8740b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:37.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.865+0000 7ff8791e9700 1 --2- 172.21.15.12:0/3817708145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8740c9970 0x7ff8740b04e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00cbc0 tx=0x7ff86c00cf80 comp rx=0 tx=0).stop 2023-07-08T12:38:37.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.865+0000 7ff8791e9700 1 -- 172.21.15.12:0/3817708145 shutdown_connections 2023-07-08T12:38:37.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.865+0000 7ff8791e9700 1 --2- 172.21.15.12:0/3817708145 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7ff86006adb0 0x7ff86006d260 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.865+0000 7ff8791e9700 1 --2- 172.21.15.12:0/3817708145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8740cb8b0 0x7ff8741feba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.865+0000 7ff8791e9700 1 --2- 172.21.15.12:0/3817708145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8740c9970 0x7ff8740b04e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.865+0000 7ff8791e9700 1 --2- 172.21.15.12:0/3817708145 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740c9d40 0x7ff8741f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:37.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.865+0000 7ff8791e9700 1 -- 172.21.15.12:0/3817708145 >> 172.21.15.12:0/3817708145 conn(0x7ff87406abc0 msgr2=0x7ff87406e300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:37.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.865+0000 7ff8791e9700 1 -- 172.21.15.12:0/3817708145 shutdown_connections 2023-07-08T12:38:37.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:37.865+0000 7ff8791e9700 1 -- 172.21.15.12:0/3817708145 wait complete. 2023-07-08T12:38:37.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.5 2023-07-08T12:38:37.879 DEBUG:teuthology.parallel:result is None 2023-07-08T12:38:37.879 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-08T12:38:37.879 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-08T12:38:38.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.043+0000 7f80dfa9f700 1 Processor -- start 2023-07-08T12:38:38.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.043+0000 7f80dfa9f700 1 -- start start 2023-07-08T12:38:38.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80dfa9f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80d8070bb0 0x7f80d815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80dfa9f700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d806b1b0 con 0x7f80d806b640 2023-07-08T12:38:38.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80dfa9f700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d815d1f0 con 0x7f80d80cd530 2023-07-08T12:38:38.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80dfa9f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d806a5a0 con 0x7f80d8070bb0 2023-07-08T12:38:38.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80de03c700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f80d806b640 0x7f80d80c9570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37390/0 (socket says 172.21.15.12:37390) 2023-07-08T12:38:38.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80de03c700 1 -- 172.21.15.12:0/4206823904 learned_addr learned my addr 172.21.15.12:0/4206823904 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:38.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80dd83b700 1 --2- 172.21.15.12:0/4206823904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80d8070bb0 0x7f80d815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77011828 0 0) 0x7f80d806b1b0 con 0x7f80d806b640 2023-07-08T12:38:38.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80c40035b0 con 0x7f80d806b640 2023-07-08T12:38:38.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3270596010 0 0) 0x7f80c40035b0 con 0x7f80d806b640 2023-07-08T12:38:38.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80d8070bb0 msgr2=0x7f80d815ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:38.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80dc839700 1 --2- 172.21.15.12:0/4206823904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80d8070bb0 0x7f80d815ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 >> v1:172.21.15.160:6789/0 conn(0x7f80d80cd530 legacy=0x7f80d8070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d806a880 con 0x7f80d806b640 2023-07-08T12:38:38.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.044+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f80d00019f0 con 0x7f80d806b640 2023-07-08T12:38:38.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 >> v1:172.21.15.12:6789/0 conn(0x7f80d806b640 legacy=0x7f80d80c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80dc839700 1 --2- 172.21.15.12:0/4206823904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80d8070bb0 0x7f80c4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80dc839700 1 --2- 172.21.15.12:0/4206823904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80c4003f70 0x7f80c4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80dc839700 1 --2- 172.21.15.12:0/4206823904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80c4004d90 0x7f80c4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d806a5a0 con 0x7f80c4004d90 2023-07-08T12:38:38.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c400ba40 con 0x7f80c4003f70 2023-07-08T12:38:38.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c400bc00 con 0x7f80d8070bb0 2023-07-08T12:38:38.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80de03c700 1 --2- 172.21.15.12:0/4206823904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80c4004d90 0x7f80c4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80dd83b700 1 --2- 172.21.15.12:0/4206823904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80d8070bb0 0x7f80c4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80dd03a700 1 --2- 172.21.15.12:0/4206823904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80c4003f70 0x7f80c4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80de03c700 1 -- 172.21.15.12:0/4206823904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80d8070bb0 msgr2=0x7f80c4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:38.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80de03c700 1 --2- 172.21.15.12:0/4206823904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80d8070bb0 0x7f80c4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80de03c700 1 -- 172.21.15.12:0/4206823904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80c4003f70 msgr2=0x7f80c4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:38.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80de03c700 1 --2- 172.21.15.12:0/4206823904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80c4003f70 0x7f80c4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80de03c700 1 -- 172.21.15.12:0/4206823904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c40035b0 con 0x7f80c4004d90 2023-07-08T12:38:38.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80dd83b700 1 --2- 172.21.15.12:0/4206823904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80d8070bb0 0x7f80c4003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:38:38.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80de03c700 1 --2- 172.21.15.12:0/4206823904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80c4004d90 0x7f80c4005200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f80d000a830 tx=0x7f80c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92d04eb108b7ea8 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:38.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0012600 con 0x7f80c4004d90 2023-07-08T12:38:38.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d0012780 con 0x7f80c4004d90 2023-07-08T12:38:38.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.045+0000 7f80dc839700 1 -- 172.21.15.12:0/4206823904 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0012b30 con 0x7f80c4004d90 2023-07-08T12:38:38.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.046+0000 7f80dfa9f700 1 -- 172.21.15.12:0/4206823904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80c4004d90 msgr2=0x7f80c4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.046+0000 7f80dfa9f700 1 --2- 172.21.15.12:0/4206823904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80c4004d90 0x7f80c4005200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f80d000a830 tx=0x7f80c4005880 comp rx=0 tx=0).stop 2023-07-08T12:38:38.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.046+0000 7f80de03c700 1 -- 172.21.15.12:0/4206823904 reap_dead start 2023-07-08T12:38:38.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.046+0000 7f80dfa9f700 1 -- 172.21.15.12:0/4206823904 shutdown_connections 2023-07-08T12:38:38.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.046+0000 7f80dfa9f700 1 -- 172.21.15.12:0/4206823904 >> 172.21.15.12:0/4206823904 conn(0x7f80d80cb9f0 msgr2=0x7f80d80cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:38.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.046+0000 7f80dfa9f700 1 -- 172.21.15.12:0/4206823904 shutdown_connections 2023-07-08T12:38:38.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.046+0000 7f80dfa9f700 1 -- 172.21.15.12:0/4206823904 wait complete. 2023-07-08T12:38:38.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.047+0000 7f80dfa9f700 1 Processor -- start 2023-07-08T12:38:38.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.047+0000 7f80dfa9f700 1 -- start start 2023-07-08T12:38:38.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dfa9f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80c4004d90 0x7f80d813c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dfa9f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d80c3c00 0x7f80d815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dfa9f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80d80c81c0 0x7f80d80c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dfa9f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80c4d40 con 0x7f80d80c3c00 2023-07-08T12:38:38.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dfa9f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80c8ba0 con 0x7f80d80c81c0 2023-07-08T12:38:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dfa9f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d81476b0 con 0x7f80c4004d90 2023-07-08T12:38:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dd03a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d80c3c00 0x7f80d815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dd03a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d80c3c00 0x7f80d815b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50004/0 (socket says 172.21.15.12:50004) 2023-07-08T12:38:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dd03a700 1 -- 172.21.15.12:0/2336524462 learned_addr learned my addr 172.21.15.12:0/2336524462 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dd83b700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80c4004d90 0x7f80d813c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80de03c700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80d80c81c0 0x7f80d80c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dd03a700 1 -- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80c4004d90 msgr2=0x7f80d813c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dd03a700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80c4004d90 0x7f80d813c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dd03a700 1 -- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80d80c81c0 msgr2=0x7f80d80c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dd03a700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80d80c81c0 0x7f80d80c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dd03a700 1 -- 172.21.15.12:0/2336524462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d813ceb0 con 0x7f80d80c3c00 2023-07-08T12:38:38.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80de03c700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80d80c81c0 0x7f80d80c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:38.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.048+0000 7f80dd03a700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d80c3c00 0x7f80d815b930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f80c800f860 tx=0x7f80c800fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:38.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.049+0000 7f80ce7fc700 1 -- 172.21.15.12:0/2336524462 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c80050d0 con 0x7f80d80c3c00 2023-07-08T12:38:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.049+0000 7f80ce7fc700 1 -- 172.21.15.12:0/2336524462 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c8005250 con 0x7f80d80c3c00 2023-07-08T12:38:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.049+0000 7f80dfa9f700 1 -- 172.21.15.12:0/2336524462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d80c40c0 con 0x7f80d80c3c00 2023-07-08T12:38:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.049+0000 7f80ce7fc700 1 -- 172.21.15.12:0/2336524462 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8003680 con 0x7f80d80c3c00 2023-07-08T12:38:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.049+0000 7f80dfa9f700 1 -- 172.21.15.12:0/2336524462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d80c4680 con 0x7f80d80c3c00 2023-07-08T12:38:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.050+0000 7f80dfa9f700 1 -- 172.21.15.12:0/2336524462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d80c4e80 con 0x7f80d80c3c00 2023-07-08T12:38:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.050+0000 7f80ce7fc700 1 -- 172.21.15.12:0/2336524462 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f80c80037e0 con 0x7f80d80c3c00 2023-07-08T12:38:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.050+0000 7f80ce7fc700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f80bc06ace0 0x7f80bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.051+0000 7f80ce7fc700 1 -- 172.21.15.12:0/2336524462 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f80c80895e0 con 0x7f80d80c3c00 2023-07-08T12:38:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.051+0000 7f80dd83b700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f80bc06ace0 0x7f80bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.051+0000 7f80dd83b700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f80bc06ace0 0x7f80bc06d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009e00 tx=0x7f80d4009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.055+0000 7f80ce7fc700 1 -- 172.21.15.12:0/2336524462 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f80d80c4e80 con 0x7f80d80c3c00 2023-07-08T12:38:38.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.247+0000 7f80dfa9f700 1 -- 172.21.15.12:0/2336524462 --> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80d806ae40 con 0x7f80bc06ace0 2023-07-08T12:38:38.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.257+0000 7f80ce7fc700 1 -- 172.21.15.12:0/2336524462 <== mgr.4103 v2:172.21.15.160:6832/550875184 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367347 (secure 0 0 0) 0x7f80d4019070 con 0x7f80bc06ace0 2023-07-08T12:38:38.262 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:38.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.260+0000 7f80dfa9f700 1 -- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f80bc06ace0 msgr2=0x7f80bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.260+0000 7f80dfa9f700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f80bc06ace0 0x7f80bc06d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009e00 tx=0x7f80d4009210 comp rx=0 tx=0).stop 2023-07-08T12:38:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.261+0000 7f80dfa9f700 1 -- 172.21.15.12:0/2336524462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d80c3c00 msgr2=0x7f80d815b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.261+0000 7f80dfa9f700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d80c3c00 0x7f80d815b930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f80c800f860 tx=0x7f80c800fc20 comp rx=0 tx=0).stop 2023-07-08T12:38:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.261+0000 7f80dfa9f700 1 -- 172.21.15.12:0/2336524462 shutdown_connections 2023-07-08T12:38:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.261+0000 7f80dfa9f700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f80bc06ace0 0x7f80bc06d190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.261+0000 7f80dfa9f700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80d80c81c0 0x7f80d80c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.261+0000 7f80dfa9f700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80c4004d90 0x7f80d813c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.261+0000 7f80dfa9f700 1 --2- 172.21.15.12:0/2336524462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d80c3c00 0x7f80d815b930 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.261+0000 7f80dfa9f700 1 -- 172.21.15.12:0/2336524462 >> 172.21.15.12:0/2336524462 conn(0x7f80d80cb9f0 msgr2=0x7f80d80cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:38.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.261+0000 7f80dfa9f700 1 -- 172.21.15.12:0/2336524462 shutdown_connections 2023-07-08T12:38:38.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.261+0000 7f80dfa9f700 1 -- 172.21.15.12:0/2336524462 wait complete. 2023-07-08T12:38:38.266 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-07-08T12:38:38.274 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2023-07-08T12:38:38.147687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594960,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219208,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755512,"statfs":{"total":773094113280,"available":772869644288,"internally_reserved":0,"allocated":3678208,"data_stored":2186496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003828"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604481+0000","last_change":"2023-07-08T12:38:08.558253+0000","last_active":"2023-07-08T12:38:13.604481+0000","last_peered":"2023-07-08T12:38:13.604481+0000","last_clean":"2023-07-08T12:38:13.604481+0000","last_became_active":"2023-07-08T12:38:08.557970+0000","last_became_peered":"2023-07-08T12:38:08.557970+0000","last_unstale":"2023-07-08T12:38:13.604481+0000","last_undegraded":"2023-07-08T12:38:13.604481+0000","last_fullsized":"2023-07-08T12:38:13.604481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:56:17.613729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.608666+0000","last_change":"2023-07-08T12:38:08.546715+0000","last_active":"2023-07-08T12:38:13.608666+0000","last_peered":"2023-07-08T12:38:13.608666+0000","last_clean":"2023-07-08T12:38:13.608666+0000","last_became_active":"2023-07-08T12:38:08.546365+0000","last_became_peered":"2023-07-08T12:38:08.546365+0000","last_unstale":"2023-07-08T12:38:13.608666+0000","last_undegraded":"2023-07-08T12:38:13.608666+0000","last_fullsized":"2023-07-08T12:38:13.608666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:55:50.492319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606777+0000","last_change":"2023-07-08T12:38:08.561903+0000","last_active":"2023-07-08T12:38:13.606777+0000","last_peered":"2023-07-08T12:38:13.606777+0000","last_clean":"2023-07-08T12:38:13.606777+0000","last_became_active":"2023-07-08T12:38:08.561533+0000","last_became_peered":"2023-07-08T12:38:08.561533+0000","last_unstale":"2023-07-08T12:38:13.606777+0000","last_undegraded":"2023-07-08T12:38:13.606777+0000","last_fullsized":"2023-07-08T12:38:13.606777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:26:30.519413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758858+0000","last_change":"2023-07-08T12:38:08.556755+0000","last_active":"2023-07-08T12:38:13.758858+0000","last_peered":"2023-07-08T12:38:13.758858+0000","last_clean":"2023-07-08T12:38:13.758858+0000","last_became_active":"2023-07-08T12:38:08.556476+0000","last_became_peered":"2023-07-08T12:38:08.556476+0000","last_unstale":"2023-07-08T12:38:13.758858+0000","last_undegraded":"2023-07-08T12:38:13.758858+0000","last_fullsized":"2023-07-08T12:38:13.758858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:52.854234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759575+0000","last_change":"2023-07-08T12:38:08.561313+0000","last_active":"2023-07-08T12:38:13.759575+0000","last_peered":"2023-07-08T12:38:13.759575+0000","last_clean":"2023-07-08T12:38:13.759575+0000","last_became_active":"2023-07-08T12:38:08.561055+0000","last_became_peered":"2023-07-08T12:38:08.561055+0000","last_unstale":"2023-07-08T12:38:13.759575+0000","last_undegraded":"2023-07-08T12:38:13.759575+0000","last_fullsized":"2023-07-08T12:38:13.759575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:49.502059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606630+0000","last_change":"2023-07-08T12:38:08.564651+0000","last_active":"2023-07-08T12:38:13.606630+0000","last_peered":"2023-07-08T12:38:13.606630+0000","last_clean":"2023-07-08T12:38:13.606630+0000","last_became_active":"2023-07-08T12:38:08.564389+0000","last_became_peered":"2023-07-08T12:38:08.564389+0000","last_unstale":"2023-07-08T12:38:13.606630+0000","last_undegraded":"2023-07-08T12:38:13.606630+0000","last_fullsized":"2023-07-08T12:38:13.606630+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:03:39.903947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758619+0000","last_change":"2023-07-08T12:38:08.569425+0000","last_active":"2023-07-08T12:38:13.758619+0000","last_peered":"2023-07-08T12:38:13.758619+0000","last_clean":"2023-07-08T12:38:13.758619+0000","last_became_active":"2023-07-08T12:38:08.569167+0000","last_became_peered":"2023-07-08T12:38:08.569167+0000","last_unstale":"2023-07-08T12:38:13.758619+0000","last_undegraded":"2023-07-08T12:38:13.758619+0000","last_fullsized":"2023-07-08T12:38:13.758619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:07.081181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605308+0000","last_change":"2023-07-08T12:38:08.549000+0000","last_active":"2023-07-08T12:38:13.605308+0000","last_peered":"2023-07-08T12:38:13.605308+0000","last_clean":"2023-07-08T12:38:13.605308+0000","last_became_active":"2023-07-08T12:38:08.548657+0000","last_became_peered":"2023-07-08T12:38:08.548657+0000","last_unstale":"2023-07-08T12:38:13.605308+0000","last_undegraded":"2023-07-08T12:38:13.605308+0000","last_fullsized":"2023-07-08T12:38:13.605308+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:37.290100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759490+0000","last_change":"2023-07-08T12:38:08.565002+0000","last_active":"2023-07-08T12:38:13.759490+0000","last_peered":"2023-07-08T12:38:13.759490+0000","last_clean":"2023-07-08T12:38:13.759490+0000","last_became_active":"2023-07-08T12:38:08.564554+0000","last_became_peered":"2023-07-08T12:38:08.564554+0000","last_unstale":"2023-07-08T12:38:13.759490+0000","last_undegraded":"2023-07-08T12:38:13.759490+0000","last_fullsized":"2023-07-08T12:38:13.759490+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:35.758126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606670+0000","last_change":"2023-07-08T12:38:08.556118+0000","last_active":"2023-07-08T12:38:13.606670+0000","last_peered":"2023-07-08T12:38:13.606670+0000","last_clean":"2023-07-08T12:38:13.606670+0000","last_became_active":"2023-07-08T12:38:08.555833+0000","last_became_peered":"2023-07-08T12:38:08.555833+0000","last_unstale":"2023-07-08T12:38:13.606670+0000","last_undegraded":"2023-07-08T12:38:13.606670+0000","last_fullsized":"2023-07-08T12:38:13.606670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:03.390028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606704+0000","last_change":"2023-07-08T12:38:08.568872+0000","last_active":"2023-07-08T12:38:13.606704+0000","last_peered":"2023-07-08T12:38:13.606704+0000","last_clean":"2023-07-08T12:38:13.606704+0000","last_became_active":"2023-07-08T12:38:08.568624+0000","last_became_peered":"2023-07-08T12:38:08.568624+0000","last_unstale":"2023-07-08T12:38:13.606704+0000","last_undegraded":"2023-07-08T12:38:13.606704+0000","last_fullsized":"2023-07-08T12:38:13.606704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:35:09.678657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604156+0000","last_change":"2023-07-08T12:38:08.560950+0000","last_active":"2023-07-08T12:38:13.604156+0000","last_peered":"2023-07-08T12:38:13.604156+0000","last_clean":"2023-07-08T12:38:13.604156+0000","last_became_active":"2023-07-08T12:38:08.560621+0000","last_became_peered":"2023-07-08T12:38:08.560621+0000","last_unstale":"2023-07-08T12:38:13.604156+0000","last_undegraded":"2023-07-08T12:38:13.604156+0000","last_fullsized":"2023-07-08T12:38:13.604156+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T12:59:32.693976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604193+0000","last_change":"2023-07-08T12:38:08.556685+0000","last_active":"2023-07-08T12:38:13.604193+0000","last_peered":"2023-07-08T12:38:13.604193+0000","last_clean":"2023-07-08T12:38:13.604193+0000","last_became_active":"2023-07-08T12:38:08.556387+0000","last_became_peered":"2023-07-08T12:38:08.556387+0000","last_unstale":"2023-07-08T12:38:13.604193+0000","last_undegraded":"2023-07-08T12:38:13.604193+0000","last_fullsized":"2023-07-08T12:38:13.604193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:03:36.626338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604952+0000","last_change":"2023-07-08T12:38:08.547070+0000","last_active":"2023-07-08T12:38:13.604952+0000","last_peered":"2023-07-08T12:38:13.604952+0000","last_clean":"2023-07-08T12:38:13.604952+0000","last_became_active":"2023-07-08T12:38:08.546813+0000","last_became_peered":"2023-07-08T12:38:08.546813+0000","last_unstale":"2023-07-08T12:38:13.604952+0000","last_undegraded":"2023-07-08T12:38:13.604952+0000","last_fullsized":"2023-07-08T12:38:13.604952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758137+0000","last_change":"2023-07-08T12:38:08.569414+0000","last_active":"2023-07-08T12:38:13.758137+0000","last_peered":"2023-07-08T12:38:13.758137+0000","last_clean":"2023-07-08T12:38:13.758137+0000","last_became_active":"2023-07-08T12:38:08.569177+0000","last_became_peered":"2023-07-08T12:38:08.569177+0000","last_unstale":"2023-07-08T12:38:13.758137+0000","last_undegraded":"2023-07-08T12:38:13.758137+0000","last_fullsized":"2023-07-08T12:38:13.758137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606000+0000","last_change":"2023-07-08T12:38:08.561730+0000","last_active":"2023-07-08T12:38:13.606000+0000","last_peered":"2023-07-08T12:38:13.606000+0000","last_clean":"2023-07-08T12:38:13.606000+0000","last_became_active":"2023-07-08T12:38:08.561498+0000","last_became_peered":"2023-07-08T12:38:08.561498+0000","last_unstale":"2023-07-08T12:38:13.606000+0000","last_undegraded":"2023-07-08T12:38:13.606000+0000","last_fullsized":"2023-07-08T12:38:13.606000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:06:06.921530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759585+0000","last_change":"2023-07-08T12:38:08.568400+0000","last_active":"2023-07-08T12:38:13.759585+0000","last_peered":"2023-07-08T12:38:13.759585+0000","last_clean":"2023-07-08T12:38:13.759585+0000","last_became_active":"2023-07-08T12:38:08.568070+0000","last_became_peered":"2023-07-08T12:38:08.568070+0000","last_unstale":"2023-07-08T12:38:13.759585+0000","last_undegraded":"2023-07-08T12:38:13.759585+0000","last_fullsized":"2023-07-08T12:38:13.759585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:35.332610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606886+0000","last_change":"2023-07-08T12:38:08.549069+0000","last_active":"2023-07-08T12:38:13.606886+0000","last_peered":"2023-07-08T12:38:13.606886+0000","last_clean":"2023-07-08T12:38:13.606886+0000","last_became_active":"2023-07-08T12:38:08.548774+0000","last_became_peered":"2023-07-08T12:38:08.548774+0000","last_unstale":"2023-07-08T12:38:13.606886+0000","last_undegraded":"2023-07-08T12:38:13.606886+0000","last_fullsized":"2023-07-08T12:38:13.606886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:02:05.917920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604270+0000","last_change":"2023-07-08T12:38:08.553458+0000","last_active":"2023-07-08T12:38:13.604270+0000","last_peered":"2023-07-08T12:38:13.604270+0000","last_clean":"2023-07-08T12:38:13.604270+0000","last_became_active":"2023-07-08T12:38:08.553177+0000","last_became_peered":"2023-07-08T12:38:08.553177+0000","last_unstale":"2023-07-08T12:38:13.604270+0000","last_undegraded":"2023-07-08T12:38:13.604270+0000","last_fullsized":"2023-07-08T12:38:13.604270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:07:08.413767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606438+0000","last_change":"2023-07-08T12:38:08.546039+0000","last_active":"2023-07-08T12:38:13.606438+0000","last_peered":"2023-07-08T12:38:13.606438+0000","last_clean":"2023-07-08T12:38:13.606438+0000","last_became_active":"2023-07-08T12:38:08.545723+0000","last_became_peered":"2023-07-08T12:38:08.545723+0000","last_unstale":"2023-07-08T12:38:13.606438+0000","last_undegraded":"2023-07-08T12:38:13.606438+0000","last_fullsized":"2023-07-08T12:38:13.606438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:05:03.136361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758578+0000","last_change":"2023-07-08T12:38:08.558172+0000","last_active":"2023-07-08T12:38:13.758578+0000","last_peered":"2023-07-08T12:38:13.758578+0000","last_clean":"2023-07-08T12:38:13.758578+0000","last_became_active":"2023-07-08T12:38:08.557888+0000","last_became_peered":"2023-07-08T12:38:08.557888+0000","last_unstale":"2023-07-08T12:38:13.758578+0000","last_undegraded":"2023-07-08T12:38:13.758578+0000","last_fullsized":"2023-07-08T12:38:13.758578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:37.290100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758835+0000","last_change":"2023-07-08T12:38:08.569776+0000","last_active":"2023-07-08T12:38:13.758835+0000","last_peered":"2023-07-08T12:38:13.758835+0000","last_clean":"2023-07-08T12:38:13.758835+0000","last_became_active":"2023-07-08T12:38:08.569530+0000","last_became_peered":"2023-07-08T12:38:08.569530+0000","last_unstale":"2023-07-08T12:38:13.758835+0000","last_undegraded":"2023-07-08T12:38:13.758835+0000","last_fullsized":"2023-07-08T12:38:13.758835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:18:27.333138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759672+0000","last_change":"2023-07-08T12:38:08.557720+0000","last_active":"2023-07-08T12:38:13.759672+0000","last_peered":"2023-07-08T12:38:13.759672+0000","last_clean":"2023-07-08T12:38:13.759672+0000","last_became_active":"2023-07-08T12:38:08.557390+0000","last_became_peered":"2023-07-08T12:38:08.557390+0000","last_unstale":"2023-07-08T12:38:13.759672+0000","last_undegraded":"2023-07-08T12:38:13.759672+0000","last_fullsized":"2023-07-08T12:38:13.759672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:13:13.582721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758123+0000","last_change":"2023-07-08T12:38:08.558048+0000","last_active":"2023-07-08T12:38:13.758123+0000","last_peered":"2023-07-08T12:38:13.758123+0000","last_clean":"2023-07-08T12:38:13.758123+0000","last_became_active":"2023-07-08T12:38:08.557871+0000","last_became_peered":"2023-07-08T12:38:08.557871+0000","last_unstale":"2023-07-08T12:38:13.758123+0000","last_undegraded":"2023-07-08T12:38:13.758123+0000","last_fullsized":"2023-07-08T12:38:13.758123+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:40.634514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759316+0000","last_change":"2023-07-08T12:38:08.553223+0000","last_active":"2023-07-08T12:38:13.759316+0000","last_peered":"2023-07-08T12:38:13.759316+0000","last_clean":"2023-07-08T12:38:13.759316+0000","last_became_active":"2023-07-08T12:38:08.552416+0000","last_became_peered":"2023-07-08T12:38:08.552416+0000","last_unstale":"2023-07-08T12:38:13.759316+0000","last_undegraded":"2023-07-08T12:38:13.759316+0000","last_fullsized":"2023-07-08T12:38:13.759316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:06:26.551781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760263+0000","last_change":"2023-07-08T12:38:08.553769+0000","last_active":"2023-07-08T12:38:13.760263+0000","last_peered":"2023-07-08T12:38:13.760263+0000","last_clean":"2023-07-08T12:38:13.760263+0000","last_became_active":"2023-07-08T12:38:08.553502+0000","last_became_peered":"2023-07-08T12:38:08.553502+0000","last_unstale":"2023-07-08T12:38:13.760263+0000","last_undegraded":"2023-07-08T12:38:13.760263+0000","last_fullsized":"2023-07-08T12:38:13.760263+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:35.758126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605466+0000","last_change":"2023-07-08T12:38:08.546308+0000","last_active":"2023-07-08T12:38:13.605466+0000","last_peered":"2023-07-08T12:38:13.605466+0000","last_clean":"2023-07-08T12:38:13.605466+0000","last_became_active":"2023-07-08T12:38:08.545865+0000","last_became_peered":"2023-07-08T12:38:08.545865+0000","last_unstale":"2023-07-08T12:38:13.605466+0000","last_undegraded":"2023-07-08T12:38:13.605466+0000","last_fullsized":"2023-07-08T12:38:13.605466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:22.667819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606681+0000","last_change":"2023-07-08T12:38:12.584534+0000","last_active":"2023-07-08T12:38:13.606681+0000","last_peered":"2023-07-08T12:38:13.606681+0000","last_clean":"2023-07-08T12:38:13.606681+0000","last_became_active":"2023-07-08T12:38:12.584153+0000","last_became_peered":"2023-07-08T12:38:12.584153+0000","last_unstale":"2023-07-08T12:38:13.606681+0000","last_undegraded":"2023-07-08T12:38:13.606681+0000","last_fullsized":"2023-07-08T12:38:13.606681+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:21:34.996680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606263+0000","last_change":"2023-07-08T12:38:07.547338+0000","last_active":"2023-07-08T12:38:13.606263+0000","last_peered":"2023-07-08T12:38:13.606263+0000","last_clean":"2023-07-08T12:38:13.606263+0000","last_became_active":"2023-07-08T12:38:07.547161+0000","last_became_peered":"2023-07-08T12:38:07.547161+0000","last_unstale":"2023-07-08T12:38:13.606263+0000","last_undegraded":"2023-07-08T12:38:13.606263+0000","last_fullsized":"2023-07-08T12:38:13.606263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:37:52.614496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759146+0000","last_change":"2023-07-08T12:38:08.553623+0000","last_active":"2023-07-08T12:38:13.759146+0000","last_peered":"2023-07-08T12:38:13.759146+0000","last_clean":"2023-07-08T12:38:13.759146+0000","last_became_active":"2023-07-08T12:38:08.551973+0000","last_became_peered":"2023-07-08T12:38:08.551973+0000","last_unstale":"2023-07-08T12:38:13.759146+0000","last_undegraded":"2023-07-08T12:38:13.759146+0000","last_fullsized":"2023-07-08T12:38:13.759146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:55:50.492319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759324+0000","last_change":"2023-07-08T12:38:12.590297+0000","last_active":"2023-07-08T12:38:13.759324+0000","last_peered":"2023-07-08T12:38:13.759324+0000","last_clean":"2023-07-08T12:38:13.759324+0000","last_became_active":"2023-07-08T12:38:12.589898+0000","last_became_peered":"2023-07-08T12:38:12.589898+0000","last_unstale":"2023-07-08T12:38:13.759324+0000","last_undegraded":"2023-07-08T12:38:13.759324+0000","last_fullsized":"2023-07-08T12:38:13.759324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:55.150008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758590+0000","last_change":"2023-07-08T12:38:07.548760+0000","last_active":"2023-07-08T12:38:13.758590+0000","last_peered":"2023-07-08T12:38:13.758590+0000","last_clean":"2023-07-08T12:38:13.758590+0000","last_became_active":"2023-07-08T12:38:07.548448+0000","last_became_peered":"2023-07-08T12:38:07.548448+0000","last_unstale":"2023-07-08T12:38:13.758590+0000","last_undegraded":"2023-07-08T12:38:13.758590+0000","last_fullsized":"2023-07-08T12:38:13.758590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T16:04:24.797809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759344+0000","last_change":"2023-07-08T12:38:08.561715+0000","last_active":"2023-07-08T12:38:13.759344+0000","last_peered":"2023-07-08T12:38:13.759344+0000","last_clean":"2023-07-08T12:38:13.759344+0000","last_became_active":"2023-07-08T12:38:08.561004+0000","last_became_peered":"2023-07-08T12:38:08.561004+0000","last_unstale":"2023-07-08T12:38:13.759344+0000","last_undegraded":"2023-07-08T12:38:13.759344+0000","last_fullsized":"2023-07-08T12:38:13.759344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:40.634514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759948+0000","last_change":"2023-07-08T12:38:12.593094+0000","last_active":"2023-07-08T12:38:13.759948+0000","last_peered":"2023-07-08T12:38:13.759948+0000","last_clean":"2023-07-08T12:38:13.759948+0000","last_became_active":"2023-07-08T12:38:12.592704+0000","last_became_peered":"2023-07-08T12:38:12.592704+0000","last_unstale":"2023-07-08T12:38:13.759948+0000","last_undegraded":"2023-07-08T12:38:13.759948+0000","last_fullsized":"2023-07-08T12:38:13.759948+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:26.710918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758339+0000","last_change":"2023-07-08T12:38:07.543437+0000","last_active":"2023-07-08T12:38:13.758339+0000","last_peered":"2023-07-08T12:38:13.758339+0000","last_clean":"2023-07-08T12:38:13.758339+0000","last_became_active":"2023-07-08T12:38:07.543158+0000","last_became_peered":"2023-07-08T12:38:07.543158+0000","last_unstale":"2023-07-08T12:38:13.758339+0000","last_undegraded":"2023-07-08T12:38:13.758339+0000","last_fullsized":"2023-07-08T12:38:13.758339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:48.497459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759270+0000","last_change":"2023-07-08T12:38:08.552907+0000","last_active":"2023-07-08T12:38:13.759270+0000","last_peered":"2023-07-08T12:38:13.759270+0000","last_clean":"2023-07-08T12:38:13.759270+0000","last_became_active":"2023-07-08T12:38:08.552286+0000","last_became_peered":"2023-07-08T12:38:08.552286+0000","last_unstale":"2023-07-08T12:38:13.759270+0000","last_undegraded":"2023-07-08T12:38:13.759270+0000","last_fullsized":"2023-07-08T12:38:13.759270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:11:32.912205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758741+0000","last_change":"2023-07-08T12:38:12.590030+0000","last_active":"2023-07-08T12:38:13.758741+0000","last_peered":"2023-07-08T12:38:13.758741+0000","last_clean":"2023-07-08T12:38:13.758741+0000","last_became_active":"2023-07-08T12:38:12.589580+0000","last_became_peered":"2023-07-08T12:38:12.589580+0000","last_unstale":"2023-07-08T12:38:13.758741+0000","last_undegraded":"2023-07-08T12:38:13.758741+0000","last_fullsized":"2023-07-08T12:38:13.758741+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:08:35.080577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759616+0000","last_change":"2023-07-08T12:38:07.541564+0000","last_active":"2023-07-08T12:38:13.759616+0000","last_peered":"2023-07-08T12:38:13.759616+0000","last_clean":"2023-07-08T12:38:13.759616+0000","last_became_active":"2023-07-08T12:38:07.540886+0000","last_became_peered":"2023-07-08T12:38:07.540886+0000","last_unstale":"2023-07-08T12:38:13.759616+0000","last_undegraded":"2023-07-08T12:38:13.759616+0000","last_fullsized":"2023-07-08T12:38:13.759616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:35.414408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758276+0000","last_change":"2023-07-08T12:38:08.556742+0000","last_active":"2023-07-08T12:38:13.758276+0000","last_peered":"2023-07-08T12:38:13.758276+0000","last_clean":"2023-07-08T12:38:13.758276+0000","last_became_active":"2023-07-08T12:38:08.556264+0000","last_became_peered":"2023-07-08T12:38:08.556264+0000","last_unstale":"2023-07-08T12:38:13.758276+0000","last_undegraded":"2023-07-08T12:38:13.758276+0000","last_fullsized":"2023-07-08T12:38:13.758276+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:56.156240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758009+0000","last_change":"2023-07-08T12:38:12.587899+0000","last_active":"2023-07-08T12:38:13.758009+0000","last_peered":"2023-07-08T12:38:13.758009+0000","last_clean":"2023-07-08T12:38:13.758009+0000","last_became_active":"2023-07-08T12:38:12.587591+0000","last_became_peered":"2023-07-08T12:38:12.587591+0000","last_unstale":"2023-07-08T12:38:13.758009+0000","last_undegraded":"2023-07-08T12:38:13.758009+0000","last_fullsized":"2023-07-08T12:38:13.758009+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:19.333590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605565+0000","last_change":"2023-07-08T12:38:07.536496+0000","last_active":"2023-07-08T12:38:13.605565+0000","last_peered":"2023-07-08T12:38:13.605565+0000","last_clean":"2023-07-08T12:38:13.605565+0000","last_became_active":"2023-07-08T12:38:07.535995+0000","last_became_peered":"2023-07-08T12:38:07.535995+0000","last_unstale":"2023-07-08T12:38:13.605565+0000","last_undegraded":"2023-07-08T12:38:13.605565+0000","last_fullsized":"2023-07-08T12:38:13.605565+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:28:48.361016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758283+0000","last_change":"2023-07-08T12:38:08.553479+0000","last_active":"2023-07-08T12:38:13.758283+0000","last_peered":"2023-07-08T12:38:13.758283+0000","last_clean":"2023-07-08T12:38:13.758283+0000","last_became_active":"2023-07-08T12:38:08.550664+0000","last_became_peered":"2023-07-08T12:38:08.550664+0000","last_unstale":"2023-07-08T12:38:13.758283+0000","last_undegraded":"2023-07-08T12:38:13.758283+0000","last_fullsized":"2023-07-08T12:38:13.758283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758974+0000","last_change":"2023-07-08T12:38:12.592991+0000","last_active":"2023-07-08T12:38:13.758974+0000","last_peered":"2023-07-08T12:38:13.758974+0000","last_clean":"2023-07-08T12:38:13.758974+0000","last_became_active":"2023-07-08T12:38:12.591995+0000","last_became_peered":"2023-07-08T12:38:12.591995+0000","last_unstale":"2023-07-08T12:38:13.758974+0000","last_undegraded":"2023-07-08T12:38:13.758974+0000","last_fullsized":"2023-07-08T12:38:13.758974+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:07.433127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759097+0000","last_change":"2023-07-08T12:38:07.551354+0000","last_active":"2023-07-08T12:38:13.759097+0000","last_peered":"2023-07-08T12:38:13.759097+0000","last_clean":"2023-07-08T12:38:13.759097+0000","last_became_active":"2023-07-08T12:38:07.551134+0000","last_became_peered":"2023-07-08T12:38:07.551134+0000","last_unstale":"2023-07-08T12:38:13.759097+0000","last_undegraded":"2023-07-08T12:38:13.759097+0000","last_fullsized":"2023-07-08T12:38:13.759097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:41:53.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760978+0000","last_change":"2023-07-08T12:38:08.558470+0000","last_active":"2023-07-08T12:38:13.760978+0000","last_peered":"2023-07-08T12:38:13.760978+0000","last_clean":"2023-07-08T12:38:13.760978+0000","last_became_active":"2023-07-08T12:38:08.557894+0000","last_became_peered":"2023-07-08T12:38:08.557894+0000","last_unstale":"2023-07-08T12:38:13.760978+0000","last_undegraded":"2023-07-08T12:38:13.760978+0000","last_fullsized":"2023-07-08T12:38:13.760978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:35.332610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.757971+0000","last_change":"2023-07-08T12:38:12.590703+0000","last_active":"2023-07-08T12:38:13.757971+0000","last_peered":"2023-07-08T12:38:13.757971+0000","last_clean":"2023-07-08T12:38:13.757971+0000","last_became_active":"2023-07-08T12:38:12.590268+0000","last_became_peered":"2023-07-08T12:38:12.590268+0000","last_unstale":"2023-07-08T12:38:13.757971+0000","last_undegraded":"2023-07-08T12:38:13.757971+0000","last_fullsized":"2023-07-08T12:38:13.757971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:35:36.579637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:23.855571+0000","last_change":"2023-07-08T12:38:07.544325+0000","last_active":"2023-07-08T12:38:23.855571+0000","last_peered":"2023-07-08T12:38:23.855571+0000","last_clean":"2023-07-08T12:38:23.855571+0000","last_became_active":"2023-07-08T12:38:07.543861+0000","last_became_peered":"2023-07-08T12:38:07.543861+0000","last_unstale":"2023-07-08T12:38:23.855571+0000","last_undegraded":"2023-07-08T12:38:23.855571+0000","last_fullsized":"2023-07-08T12:38:23.855571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:40:52.689398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605326+0000","last_change":"2023-07-08T12:38:08.553055+0000","last_active":"2023-07-08T12:38:13.605326+0000","last_peered":"2023-07-08T12:38:13.605326+0000","last_clean":"2023-07-08T12:38:13.605326+0000","last_became_active":"2023-07-08T12:38:08.552632+0000","last_became_peered":"2023-07-08T12:38:08.552632+0000","last_unstale":"2023-07-08T12:38:13.605326+0000","last_undegraded":"2023-07-08T12:38:13.605326+0000","last_fullsized":"2023-07-08T12:38:13.605326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:37:47.885139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605700+0000","last_change":"2023-07-08T12:38:12.598190+0000","last_active":"2023-07-08T12:38:13.605700+0000","last_peered":"2023-07-08T12:38:13.605700+0000","last_clean":"2023-07-08T12:38:13.605700+0000","last_became_active":"2023-07-08T12:38:12.597813+0000","last_became_peered":"2023-07-08T12:38:12.597813+0000","last_unstale":"2023-07-08T12:38:13.605700+0000","last_undegraded":"2023-07-08T12:38:13.605700+0000","last_fullsized":"2023-07-08T12:38:13.605700+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:41:58.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758412+0000","last_change":"2023-07-08T12:38:07.543441+0000","last_active":"2023-07-08T12:38:13.758412+0000","last_peered":"2023-07-08T12:38:13.758412+0000","last_clean":"2023-07-08T12:38:13.758412+0000","last_became_active":"2023-07-08T12:38:07.543188+0000","last_became_peered":"2023-07-08T12:38:07.543188+0000","last_unstale":"2023-07-08T12:38:13.758412+0000","last_undegraded":"2023-07-08T12:38:13.758412+0000","last_fullsized":"2023-07-08T12:38:13.758412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:34.328010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758440+0000","last_change":"2023-07-08T12:38:08.552797+0000","last_active":"2023-07-08T12:38:13.758440+0000","last_peered":"2023-07-08T12:38:13.758440+0000","last_clean":"2023-07-08T12:38:13.758440+0000","last_became_active":"2023-07-08T12:38:08.552289+0000","last_became_peered":"2023-07-08T12:38:08.552289+0000","last_unstale":"2023-07-08T12:38:13.758440+0000","last_undegraded":"2023-07-08T12:38:13.758440+0000","last_fullsized":"2023-07-08T12:38:13.758440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:07.081181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758345+0000","last_change":"2023-07-08T12:38:12.592793+0000","last_active":"2023-07-08T12:38:13.758345+0000","last_peered":"2023-07-08T12:38:13.758345+0000","last_clean":"2023-07-08T12:38:13.758345+0000","last_became_active":"2023-07-08T12:38:12.592414+0000","last_became_peered":"2023-07-08T12:38:12.592414+0000","last_unstale":"2023-07-08T12:38:13.758345+0000","last_undegraded":"2023-07-08T12:38:13.758345+0000","last_fullsized":"2023-07-08T12:38:13.758345+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:39.801225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758616+0000","last_change":"2023-07-08T12:38:07.538612+0000","last_active":"2023-07-08T12:38:13.758616+0000","last_peered":"2023-07-08T12:38:13.758616+0000","last_clean":"2023-07-08T12:38:13.758616+0000","last_became_active":"2023-07-08T12:38:07.538048+0000","last_became_peered":"2023-07-08T12:38:07.538048+0000","last_unstale":"2023-07-08T12:38:13.758616+0000","last_undegraded":"2023-07-08T12:38:13.758616+0000","last_fullsized":"2023-07-08T12:38:13.758616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:04:41.955662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605266+0000","last_change":"2023-07-08T12:38:08.546131+0000","last_active":"2023-07-08T12:38:13.605266+0000","last_peered":"2023-07-08T12:38:13.605266+0000","last_clean":"2023-07-08T12:38:13.605266+0000","last_became_active":"2023-07-08T12:38:08.545491+0000","last_became_peered":"2023-07-08T12:38:08.545491+0000","last_unstale":"2023-07-08T12:38:13.605266+0000","last_undegraded":"2023-07-08T12:38:13.605266+0000","last_fullsized":"2023-07-08T12:38:13.605266+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:36.419008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758428+0000","last_change":"2023-07-08T12:38:12.584683+0000","last_active":"2023-07-08T12:38:13.758428+0000","last_peered":"2023-07-08T12:38:13.758428+0000","last_clean":"2023-07-08T12:38:13.758428+0000","last_became_active":"2023-07-08T12:38:12.584256+0000","last_became_peered":"2023-07-08T12:38:12.584256+0000","last_unstale":"2023-07-08T12:38:13.758428+0000","last_undegraded":"2023-07-08T12:38:13.758428+0000","last_fullsized":"2023-07-08T12:38:13.758428+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:41.333199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758406+0000","last_change":"2023-07-08T12:38:07.543388+0000","last_active":"2023-07-08T12:38:13.758406+0000","last_peered":"2023-07-08T12:38:13.758406+0000","last_clean":"2023-07-08T12:38:13.758406+0000","last_became_active":"2023-07-08T12:38:07.543117+0000","last_became_peered":"2023-07-08T12:38:07.543117+0000","last_unstale":"2023-07-08T12:38:13.758406+0000","last_undegraded":"2023-07-08T12:38:13.758406+0000","last_fullsized":"2023-07-08T12:38:13.758406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:21.663219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759864+0000","last_change":"2023-07-08T12:38:08.561895+0000","last_active":"2023-07-08T12:38:13.759864+0000","last_peered":"2023-07-08T12:38:13.759864+0000","last_clean":"2023-07-08T12:38:13.759864+0000","last_became_active":"2023-07-08T12:38:08.561428+0000","last_became_peered":"2023-07-08T12:38:08.561428+0000","last_unstale":"2023-07-08T12:38:13.759864+0000","last_undegraded":"2023-07-08T12:38:13.759864+0000","last_fullsized":"2023-07-08T12:38:13.759864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:07.081181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605310+0000","last_change":"2023-07-08T12:38:12.592292+0000","last_active":"2023-07-08T12:38:13.605310+0000","last_peered":"2023-07-08T12:38:13.605310+0000","last_clean":"2023-07-08T12:38:13.605310+0000","last_became_active":"2023-07-08T12:38:12.591964+0000","last_became_peered":"2023-07-08T12:38:12.591964+0000","last_unstale":"2023-07-08T12:38:13.605310+0000","last_undegraded":"2023-07-08T12:38:13.605310+0000","last_fullsized":"2023-07-08T12:38:13.605310+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:53.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758158+0000","last_change":"2023-07-08T12:38:07.543683+0000","last_active":"2023-07-08T12:38:13.758158+0000","last_peered":"2023-07-08T12:38:13.758158+0000","last_clean":"2023-07-08T12:38:13.758158+0000","last_became_active":"2023-07-08T12:38:07.543416+0000","last_became_peered":"2023-07-08T12:38:07.543416+0000","last_unstale":"2023-07-08T12:38:13.758158+0000","last_undegraded":"2023-07-08T12:38:13.758158+0000","last_fullsized":"2023-07-08T12:38:13.758158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:56:16.609129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758366+0000","last_change":"2023-07-08T12:38:08.556977+0000","last_active":"2023-07-08T12:38:13.758366+0000","last_peered":"2023-07-08T12:38:13.758366+0000","last_clean":"2023-07-08T12:38:13.758366+0000","last_became_active":"2023-07-08T12:38:08.555976+0000","last_became_peered":"2023-07-08T12:38:08.555976+0000","last_unstale":"2023-07-08T12:38:13.758366+0000","last_undegraded":"2023-07-08T12:38:13.758366+0000","last_fullsized":"2023-07-08T12:38:13.758366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:36.419008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759039+0000","last_change":"2023-07-08T12:38:12.582526+0000","last_active":"2023-07-08T12:38:13.759039+0000","last_peered":"2023-07-08T12:38:13.759039+0000","last_clean":"2023-07-08T12:38:13.759039+0000","last_became_active":"2023-07-08T12:38:12.582130+0000","last_became_peered":"2023-07-08T12:38:12.582130+0000","last_unstale":"2023-07-08T12:38:13.759039+0000","last_undegraded":"2023-07-08T12:38:13.759039+0000","last_fullsized":"2023-07-08T12:38:13.759039+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:31:58.033845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759843+0000","last_change":"2023-07-08T12:38:07.543929+0000","last_active":"2023-07-08T12:38:13.759843+0000","last_peered":"2023-07-08T12:38:13.759843+0000","last_clean":"2023-07-08T12:38:13.759843+0000","last_became_active":"2023-07-08T12:38:07.543693+0000","last_became_peered":"2023-07-08T12:38:07.543693+0000","last_unstale":"2023-07-08T12:38:13.759843+0000","last_undegraded":"2023-07-08T12:38:13.759843+0000","last_fullsized":"2023-07-08T12:38:13.759843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:55:49.487719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604436+0000","last_change":"2023-07-08T12:38:08.548856+0000","last_active":"2023-07-08T12:38:13.604436+0000","last_peered":"2023-07-08T12:38:13.604436+0000","last_clean":"2023-07-08T12:38:13.604436+0000","last_became_active":"2023-07-08T12:38:08.548294+0000","last_became_peered":"2023-07-08T12:38:08.548294+0000","last_unstale":"2023-07-08T12:38:13.604436+0000","last_undegraded":"2023-07-08T12:38:13.604436+0000","last_fullsized":"2023-07-08T12:38:13.604436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:53.873880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.603862+0000","last_change":"2023-07-08T12:38:12.587147+0000","last_active":"2023-07-08T12:38:13.603862+0000","last_peered":"2023-07-08T12:38:13.603862+0000","last_clean":"2023-07-08T12:38:13.603862+0000","last_became_active":"2023-07-08T12:38:12.586709+0000","last_became_peered":"2023-07-08T12:38:12.586709+0000","last_unstale":"2023-07-08T12:38:13.603862+0000","last_undegraded":"2023-07-08T12:38:13.603862+0000","last_fullsized":"2023-07-08T12:38:13.603862+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T12:58:56.632605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759614+0000","last_change":"2023-07-08T12:38:07.545521+0000","last_active":"2023-07-08T12:38:13.759614+0000","last_peered":"2023-07-08T12:38:13.759614+0000","last_clean":"2023-07-08T12:38:13.759614+0000","last_became_active":"2023-07-08T12:38:07.545260+0000","last_became_peered":"2023-07-08T12:38:07.545260+0000","last_unstale":"2023-07-08T12:38:13.759614+0000","last_undegraded":"2023-07-08T12:38:13.759614+0000","last_fullsized":"2023-07-08T12:38:13.759614+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:02.385428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605419+0000","last_change":"2023-07-08T12:38:08.552222+0000","last_active":"2023-07-08T12:38:13.605419+0000","last_peered":"2023-07-08T12:38:13.605419+0000","last_clean":"2023-07-08T12:38:13.605419+0000","last_became_active":"2023-07-08T12:38:08.551940+0000","last_became_peered":"2023-07-08T12:38:08.551940+0000","last_unstale":"2023-07-08T12:38:13.605419+0000","last_undegraded":"2023-07-08T12:38:13.605419+0000","last_fullsized":"2023-07-08T12:38:13.605419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:31:34.141462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604126+0000","last_change":"2023-07-08T12:38:12.589202+0000","last_active":"2023-07-08T12:38:13.604126+0000","last_peered":"2023-07-08T12:38:13.604126+0000","last_clean":"2023-07-08T12:38:13.604126+0000","last_became_active":"2023-07-08T12:38:12.586775+0000","last_became_peered":"2023-07-08T12:38:12.586775+0000","last_unstale":"2023-07-08T12:38:13.604126+0000","last_undegraded":"2023-07-08T12:38:13.604126+0000","last_fullsized":"2023-07-08T12:38:13.604126+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:17:35.606731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758901+0000","last_change":"2023-07-08T12:38:07.542576+0000","last_active":"2023-07-08T12:38:13.758901+0000","last_peered":"2023-07-08T12:38:13.758901+0000","last_clean":"2023-07-08T12:38:13.758901+0000","last_became_active":"2023-07-08T12:38:07.542296+0000","last_became_peered":"2023-07-08T12:38:07.542296+0000","last_unstale":"2023-07-08T12:38:13.758901+0000","last_undegraded":"2023-07-08T12:38:13.758901+0000","last_fullsized":"2023-07-08T12:38:13.758901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:28:48.361016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758718+0000","last_change":"2023-07-08T12:38:08.560794+0000","last_active":"2023-07-08T12:38:13.758718+0000","last_peered":"2023-07-08T12:38:13.758718+0000","last_clean":"2023-07-08T12:38:13.758718+0000","last_became_active":"2023-07-08T12:38:08.560405+0000","last_became_peered":"2023-07-08T12:38:08.560405+0000","last_unstale":"2023-07-08T12:38:13.758718+0000","last_undegraded":"2023-07-08T12:38:13.758718+0000","last_fullsized":"2023-07-08T12:38:13.758718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:52:43.348476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604255+0000","last_change":"2023-07-08T12:38:12.585051+0000","last_active":"2023-07-08T12:38:13.604255+0000","last_peered":"2023-07-08T12:38:13.604255+0000","last_clean":"2023-07-08T12:38:13.604255+0000","last_became_active":"2023-07-08T12:38:12.583082+0000","last_became_peered":"2023-07-08T12:38:12.583082+0000","last_unstale":"2023-07-08T12:38:13.604255+0000","last_undegraded":"2023-07-08T12:38:13.604255+0000","last_fullsized":"2023-07-08T12:38:13.604255+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:02:43.298466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604866+0000","last_change":"2023-07-08T12:38:07.537900+0000","last_active":"2023-07-08T12:38:13.604866+0000","last_peered":"2023-07-08T12:38:13.604866+0000","last_clean":"2023-07-08T12:38:13.604866+0000","last_became_active":"2023-07-08T12:38:07.536896+0000","last_became_peered":"2023-07-08T12:38:07.536896+0000","last_unstale":"2023-07-08T12:38:13.604866+0000","last_undegraded":"2023-07-08T12:38:13.604866+0000","last_fullsized":"2023-07-08T12:38:13.604866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:55.151640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759269+0000","last_change":"2023-07-08T12:38:08.569854+0000","last_active":"2023-07-08T12:38:13.759269+0000","last_peered":"2023-07-08T12:38:13.759269+0000","last_clean":"2023-07-08T12:38:13.759269+0000","last_became_active":"2023-07-08T12:38:08.569549+0000","last_became_peered":"2023-07-08T12:38:08.569549+0000","last_unstale":"2023-07-08T12:38:13.759269+0000","last_undegraded":"2023-07-08T12:38:13.759269+0000","last_fullsized":"2023-07-08T12:38:13.759269+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:15.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604571+0000","last_change":"2023-07-08T12:38:12.580215+0000","last_active":"2023-07-08T12:38:13.604571+0000","last_peered":"2023-07-08T12:38:13.604571+0000","last_clean":"2023-07-08T12:38:13.604571+0000","last_became_active":"2023-07-08T12:38:12.579706+0000","last_became_peered":"2023-07-08T12:38:12.579706+0000","last_unstale":"2023-07-08T12:38:13.604571+0000","last_undegraded":"2023-07-08T12:38:13.604571+0000","last_fullsized":"2023-07-08T12:38:13.604571+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:08:35.080577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758601+0000","last_change":"2023-07-08T12:38:07.542376+0000","last_active":"2023-07-08T12:38:13.758601+0000","last_peered":"2023-07-08T12:38:13.758601+0000","last_clean":"2023-07-08T12:38:13.758601+0000","last_became_active":"2023-07-08T12:38:07.542046+0000","last_became_peered":"2023-07-08T12:38:07.542046+0000","last_unstale":"2023-07-08T12:38:13.758601+0000","last_undegraded":"2023-07-08T12:38:13.758601+0000","last_fullsized":"2023-07-08T12:38:13.758601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:28:48.361016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605011+0000","last_change":"2023-07-08T12:38:08.553132+0000","last_active":"2023-07-08T12:38:13.605011+0000","last_peered":"2023-07-08T12:38:13.605011+0000","last_clean":"2023-07-08T12:38:13.605011+0000","last_became_active":"2023-07-08T12:38:08.552711+0000","last_became_peered":"2023-07-08T12:38:08.552711+0000","last_unstale":"2023-07-08T12:38:13.605011+0000","last_undegraded":"2023-07-08T12:38:13.605011+0000","last_fullsized":"2023-07-08T12:38:13.605011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T16:24:56.036654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759123+0000","last_change":"2023-07-08T12:38:12.597633+0000","last_active":"2023-07-08T12:38:13.759123+0000","last_peered":"2023-07-08T12:38:13.759123+0000","last_clean":"2023-07-08T12:38:13.759123+0000","last_became_active":"2023-07-08T12:38:12.597211+0000","last_became_peered":"2023-07-08T12:38:12.597211+0000","last_unstale":"2023-07-08T12:38:13.759123+0000","last_undegraded":"2023-07-08T12:38:13.759123+0000","last_fullsized":"2023-07-08T12:38:13.759123+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:04:47.003361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758887+0000","last_change":"2023-07-08T12:38:07.542040+0000","last_active":"2023-07-08T12:38:13.758887+0000","last_peered":"2023-07-08T12:38:13.758887+0000","last_clean":"2023-07-08T12:38:13.758887+0000","last_became_active":"2023-07-08T12:38:07.541743+0000","last_became_peered":"2023-07-08T12:38:07.541743+0000","last_unstale":"2023-07-08T12:38:13.758887+0000","last_undegraded":"2023-07-08T12:38:13.758887+0000","last_fullsized":"2023-07-08T12:38:13.758887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:41:49.220614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606520+0000","last_change":"2023-07-08T12:38:08.548025+0000","last_active":"2023-07-08T12:38:13.606520+0000","last_peered":"2023-07-08T12:38:13.606520+0000","last_clean":"2023-07-08T12:38:13.606520+0000","last_became_active":"2023-07-08T12:38:08.547723+0000","last_became_peered":"2023-07-08T12:38:08.547723+0000","last_unstale":"2023-07-08T12:38:13.606520+0000","last_undegraded":"2023-07-08T12:38:13.606520+0000","last_fullsized":"2023-07-08T12:38:13.606520+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:01:21.861852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605658+0000","last_change":"2023-07-08T12:38:12.587540+0000","last_active":"2023-07-08T12:38:13.605658+0000","last_peered":"2023-07-08T12:38:13.605658+0000","last_clean":"2023-07-08T12:38:13.605658+0000","last_became_active":"2023-07-08T12:38:12.587180+0000","last_became_peered":"2023-07-08T12:38:12.587180+0000","last_unstale":"2023-07-08T12:38:13.605658+0000","last_undegraded":"2023-07-08T12:38:13.605658+0000","last_fullsized":"2023-07-08T12:38:13.605658+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:31:58.033845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605007+0000","last_change":"2023-07-08T12:38:07.535635+0000","last_active":"2023-07-08T12:38:13.605007+0000","last_peered":"2023-07-08T12:38:13.605007+0000","last_clean":"2023-07-08T12:38:13.605007+0000","last_became_active":"2023-07-08T12:38:07.535335+0000","last_became_peered":"2023-07-08T12:38:07.535335+0000","last_unstale":"2023-07-08T12:38:13.605007+0000","last_undegraded":"2023-07-08T12:38:13.605007+0000","last_fullsized":"2023-07-08T12:38:13.605007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:15:42.329406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606965+0000","last_change":"2023-07-08T12:38:08.553024+0000","last_active":"2023-07-08T12:38:13.606965+0000","last_peered":"2023-07-08T12:38:13.606965+0000","last_clean":"2023-07-08T12:38:13.606965+0000","last_became_active":"2023-07-08T12:38:08.552696+0000","last_became_peered":"2023-07-08T12:38:08.552696+0000","last_unstale":"2023-07-08T12:38:13.606965+0000","last_undegraded":"2023-07-08T12:38:13.606965+0000","last_fullsized":"2023-07-08T12:38:13.606965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:33:05.815083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606715+0000","last_change":"2023-07-08T12:38:12.584493+0000","last_active":"2023-07-08T12:38:13.606715+0000","last_peered":"2023-07-08T12:38:13.606715+0000","last_clean":"2023-07-08T12:38:13.606715+0000","last_became_active":"2023-07-08T12:38:12.584087+0000","last_became_peered":"2023-07-08T12:38:12.584087+0000","last_unstale":"2023-07-08T12:38:13.606715+0000","last_undegraded":"2023-07-08T12:38:13.606715+0000","last_fullsized":"2023-07-08T12:38:13.606715+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:10:54.808570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758207+0000","last_change":"2023-07-08T12:38:07.540092+0000","last_active":"2023-07-08T12:38:13.758207+0000","last_peered":"2023-07-08T12:38:13.758207+0000","last_clean":"2023-07-08T12:38:13.758207+0000","last_became_active":"2023-07-08T12:38:07.539789+0000","last_became_peered":"2023-07-08T12:38:07.539789+0000","last_unstale":"2023-07-08T12:38:13.758207+0000","last_undegraded":"2023-07-08T12:38:13.758207+0000","last_fullsized":"2023-07-08T12:38:13.758207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:39.629914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759601+0000","last_change":"2023-07-08T12:38:08.564633+0000","last_active":"2023-07-08T12:38:13.759601+0000","last_peered":"2023-07-08T12:38:13.759601+0000","last_clean":"2023-07-08T12:38:13.759601+0000","last_became_active":"2023-07-08T12:38:08.564323+0000","last_became_peered":"2023-07-08T12:38:08.564323+0000","last_unstale":"2023-07-08T12:38:13.759601+0000","last_undegraded":"2023-07-08T12:38:13.759601+0000","last_fullsized":"2023-07-08T12:38:13.759601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:28.004982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759180+0000","last_change":"2023-07-08T12:38:12.586250+0000","last_active":"2023-07-08T12:38:13.759180+0000","last_peered":"2023-07-08T12:38:13.759180+0000","last_clean":"2023-07-08T12:38:13.759180+0000","last_became_active":"2023-07-08T12:38:12.585838+0000","last_became_peered":"2023-07-08T12:38:12.585838+0000","last_unstale":"2023-07-08T12:38:13.759180+0000","last_undegraded":"2023-07-08T12:38:13.759180+0000","last_fullsized":"2023-07-08T12:38:13.759180+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:06:30.594880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759317+0000","last_change":"2023-07-08T12:38:03.160177+0000","last_active":"2023-07-08T12:38:13.759317+0000","last_peered":"2023-07-08T12:38:13.759317+0000","last_clean":"2023-07-08T12:38:13.759317+0000","last_became_active":"2023-07-08T12:38:03.159864+0000","last_became_peered":"2023-07-08T12:38:03.159864+0000","last_unstale":"2023-07-08T12:38:13.759317+0000","last_undegraded":"2023-07-08T12:38:13.759317+0000","last_fullsized":"2023-07-08T12:38:13.759317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:31:56.449203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604812+0000","last_change":"2023-07-08T12:38:07.537328+0000","last_active":"2023-07-08T12:38:13.604812+0000","last_peered":"2023-07-08T12:38:13.604812+0000","last_clean":"2023-07-08T12:38:13.604812+0000","last_became_active":"2023-07-08T12:38:07.537038+0000","last_became_peered":"2023-07-08T12:38:07.537038+0000","last_unstale":"2023-07-08T12:38:13.604812+0000","last_undegraded":"2023-07-08T12:38:13.604812+0000","last_fullsized":"2023-07-08T12:38:13.604812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:39.629914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760774+0000","last_change":"2023-07-08T12:38:08.558271+0000","last_active":"2023-07-08T12:38:13.760774+0000","last_peered":"2023-07-08T12:38:13.760774+0000","last_clean":"2023-07-08T12:38:13.760774+0000","last_became_active":"2023-07-08T12:38:08.557598+0000","last_became_peered":"2023-07-08T12:38:08.557598+0000","last_unstale":"2023-07-08T12:38:13.760774+0000","last_undegraded":"2023-07-08T12:38:13.760774+0000","last_fullsized":"2023-07-08T12:38:13.760774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:49.502059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759252+0000","last_change":"2023-07-08T12:38:12.591995+0000","last_active":"2023-07-08T12:38:13.759252+0000","last_peered":"2023-07-08T12:38:13.759252+0000","last_clean":"2023-07-08T12:38:13.759252+0000","last_became_active":"2023-07-08T12:38:12.591624+0000","last_became_peered":"2023-07-08T12:38:12.591624+0000","last_unstale":"2023-07-08T12:38:13.759252+0000","last_undegraded":"2023-07-08T12:38:13.759252+0000","last_fullsized":"2023-07-08T12:38:13.759252+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:57.916979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605001+0000","last_change":"2023-07-08T12:38:02.384639+0000","last_active":"2023-07-08T12:38:13.605001+0000","last_peered":"2023-07-08T12:38:13.605001+0000","last_clean":"2023-07-08T12:38:13.605001+0000","last_became_active":"2023-07-08T12:38:02.384372+0000","last_became_peered":"2023-07-08T12:38:02.384372+0000","last_unstale":"2023-07-08T12:38:13.605001+0000","last_undegraded":"2023-07-08T12:38:13.605001+0000","last_fullsized":"2023-07-08T12:38:13.605001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:29.599162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605155+0000","last_change":"2023-07-08T12:38:07.541393+0000","last_active":"2023-07-08T12:38:13.605155+0000","last_peered":"2023-07-08T12:38:13.605155+0000","last_clean":"2023-07-08T12:38:13.605155+0000","last_became_active":"2023-07-08T12:38:07.541155+0000","last_became_peered":"2023-07-08T12:38:07.541155+0000","last_unstale":"2023-07-08T12:38:13.605155+0000","last_undegraded":"2023-07-08T12:38:13.605155+0000","last_fullsized":"2023-07-08T12:38:13.605155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:51:13.319094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759210+0000","last_change":"2023-07-08T12:38:08.550498+0000","last_active":"2023-07-08T12:38:13.759210+0000","last_peered":"2023-07-08T12:38:13.759210+0000","last_clean":"2023-07-08T12:38:13.759210+0000","last_became_active":"2023-07-08T12:38:08.550106+0000","last_became_peered":"2023-07-08T12:38:08.550106+0000","last_unstale":"2023-07-08T12:38:13.759210+0000","last_undegraded":"2023-07-08T12:38:13.759210+0000","last_fullsized":"2023-07-08T12:38:13.759210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:18:27.333138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604527+0000","last_change":"2023-07-08T12:38:12.586451+0000","last_active":"2023-07-08T12:38:13.604527+0000","last_peered":"2023-07-08T12:38:13.604527+0000","last_clean":"2023-07-08T12:38:13.604527+0000","last_became_active":"2023-07-08T12:38:12.582823+0000","last_became_peered":"2023-07-08T12:38:12.582823+0000","last_unstale":"2023-07-08T12:38:13.604527+0000","last_undegraded":"2023-07-08T12:38:13.604527+0000","last_fullsized":"2023-07-08T12:38:13.604527+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:18:34.491455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604236+0000","last_change":"2023-07-08T12:38:03.212466+0000","last_active":"2023-07-08T12:38:13.604236+0000","last_peered":"2023-07-08T12:38:13.604236+0000","last_clean":"2023-07-08T12:38:13.604236+0000","last_became_active":"2023-07-08T12:38:03.212182+0000","last_became_peered":"2023-07-08T12:38:03.212182+0000","last_unstale":"2023-07-08T12:38:13.604236+0000","last_undegraded":"2023-07-08T12:38:13.604236+0000","last_fullsized":"2023-07-08T12:38:13.604236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:06:34.067771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758091+0000","last_change":"2023-07-08T12:38:07.538415+0000","last_active":"2023-07-08T12:38:13.758091+0000","last_peered":"2023-07-08T12:38:13.758091+0000","last_clean":"2023-07-08T12:38:13.758091+0000","last_became_active":"2023-07-08T12:38:07.537871+0000","last_became_peered":"2023-07-08T12:38:07.537871+0000","last_unstale":"2023-07-08T12:38:13.758091+0000","last_undegraded":"2023-07-08T12:38:13.758091+0000","last_fullsized":"2023-07-08T12:38:13.758091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:00:59.245798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759478+0000","last_change":"2023-07-08T12:38:08.554865+0000","last_active":"2023-07-08T12:38:13.759478+0000","last_peered":"2023-07-08T12:38:13.759478+0000","last_clean":"2023-07-08T12:38:13.759478+0000","last_became_active":"2023-07-08T12:38:08.554591+0000","last_became_peered":"2023-07-08T12:38:08.554591+0000","last_unstale":"2023-07-08T12:38:13.759478+0000","last_undegraded":"2023-07-08T12:38:13.759478+0000","last_fullsized":"2023-07-08T12:38:13.759478+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:53.873880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758470+0000","last_change":"2023-07-08T12:38:12.579919+0000","last_active":"2023-07-08T12:38:13.758470+0000","last_peered":"2023-07-08T12:38:13.758470+0000","last_clean":"2023-07-08T12:38:13.758470+0000","last_became_active":"2023-07-08T12:38:12.579500+0000","last_became_peered":"2023-07-08T12:38:12.579500+0000","last_unstale":"2023-07-08T12:38:13.758470+0000","last_undegraded":"2023-07-08T12:38:13.758470+0000","last_fullsized":"2023-07-08T12:38:13.758470+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:53.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605571+0000","last_change":"2023-07-08T12:38:02.380175+0000","last_active":"2023-07-08T12:38:13.605571+0000","last_peered":"2023-07-08T12:38:13.605571+0000","last_clean":"2023-07-08T12:38:13.605571+0000","last_became_active":"2023-07-08T12:38:02.379867+0000","last_became_peered":"2023-07-08T12:38:02.379867+0000","last_unstale":"2023-07-08T12:38:13.605571+0000","last_undegraded":"2023-07-08T12:38:13.605571+0000","last_fullsized":"2023-07-08T12:38:13.605571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:35:57.492852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758117+0000","last_change":"2023-07-08T12:37:58.222371+0000","last_active":"2023-07-08T12:38:13.758117+0000","last_peered":"2023-07-08T12:38:13.758117+0000","last_clean":"2023-07-08T12:38:13.758117+0000","last_became_active":"2023-07-08T12:37:58.221381+0000","last_became_peered":"2023-07-08T12:37:58.221381+0000","last_unstale":"2023-07-08T12:38:13.758117+0000","last_undegraded":"2023-07-08T12:38:13.758117+0000","last_fullsized":"2023-07-08T12:38:13.758117+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:37:56.327863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:37:56.327863+0000","last_clean_scrub_stamp":"2023-07-08T12:37:56.327863+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:44.968733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758891+0000","last_change":"2023-07-08T12:38:07.536547+0000","last_active":"2023-07-08T12:38:13.758891+0000","last_peered":"2023-07-08T12:38:13.758891+0000","last_clean":"2023-07-08T12:38:13.758891+0000","last_became_active":"2023-07-08T12:38:07.536255+0000","last_became_peered":"2023-07-08T12:38:07.536255+0000","last_unstale":"2023-07-08T12:38:13.758891+0000","last_undegraded":"2023-07-08T12:38:13.758891+0000","last_fullsized":"2023-07-08T12:38:13.758891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:41:53.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759285+0000","last_change":"2023-07-08T12:38:08.550673+0000","last_active":"2023-07-08T12:38:13.759285+0000","last_peered":"2023-07-08T12:38:13.759285+0000","last_clean":"2023-07-08T12:38:13.759285+0000","last_became_active":"2023-07-08T12:38:08.550347+0000","last_became_peered":"2023-07-08T12:38:08.550347+0000","last_unstale":"2023-07-08T12:38:13.759285+0000","last_undegraded":"2023-07-08T12:38:13.759285+0000","last_fullsized":"2023-07-08T12:38:13.759285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:03:34.056111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605442+0000","last_change":"2023-07-08T12:38:12.597251+0000","last_active":"2023-07-08T12:38:13.605442+0000","last_peered":"2023-07-08T12:38:13.605442+0000","last_clean":"2023-07-08T12:38:13.605442+0000","last_became_active":"2023-07-08T12:38:12.596856+0000","last_became_peered":"2023-07-08T12:38:12.596856+0000","last_unstale":"2023-07-08T12:38:13.605442+0000","last_undegraded":"2023-07-08T12:38:13.605442+0000","last_fullsized":"2023-07-08T12:38:13.605442+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:11.124280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758299+0000","last_change":"2023-07-08T12:38:02.385193+0000","last_active":"2023-07-08T12:38:13.758299+0000","last_peered":"2023-07-08T12:38:13.758299+0000","last_clean":"2023-07-08T12:38:13.758299+0000","last_became_active":"2023-07-08T12:38:02.384865+0000","last_became_peered":"2023-07-08T12:38:02.384865+0000","last_unstale":"2023-07-08T12:38:13.758299+0000","last_undegraded":"2023-07-08T12:38:13.758299+0000","last_fullsized":"2023-07-08T12:38:13.758299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:30.260044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.757707+0000","last_change":"2023-07-08T12:38:07.543452+0000","last_active":"2023-07-08T12:38:13.757707+0000","last_peered":"2023-07-08T12:38:13.757707+0000","last_clean":"2023-07-08T12:38:13.757707+0000","last_became_active":"2023-07-08T12:38:07.543141+0000","last_became_peered":"2023-07-08T12:38:07.543141+0000","last_unstale":"2023-07-08T12:38:13.757707+0000","last_undegraded":"2023-07-08T12:38:13.757707+0000","last_fullsized":"2023-07-08T12:38:13.757707+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:03:16.870782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759081+0000","last_change":"2023-07-08T12:38:08.570468+0000","last_active":"2023-07-08T12:38:13.759081+0000","last_peered":"2023-07-08T12:38:13.759081+0000","last_clean":"2023-07-08T12:38:13.759081+0000","last_became_active":"2023-07-08T12:38:08.570183+0000","last_became_peered":"2023-07-08T12:38:08.570183+0000","last_unstale":"2023-07-08T12:38:13.759081+0000","last_undegraded":"2023-07-08T12:38:13.759081+0000","last_fullsized":"2023-07-08T12:38:13.759081+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:32:02.608167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759172+0000","last_change":"2023-07-08T12:38:12.597186+0000","last_active":"2023-07-08T12:38:13.759172+0000","last_peered":"2023-07-08T12:38:13.759172+0000","last_clean":"2023-07-08T12:38:13.759172+0000","last_became_active":"2023-07-08T12:38:12.596772+0000","last_became_peered":"2023-07-08T12:38:12.596772+0000","last_unstale":"2023-07-08T12:38:13.759172+0000","last_undegraded":"2023-07-08T12:38:13.759172+0000","last_fullsized":"2023-07-08T12:38:13.759172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:06:56.291254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758758+0000","last_change":"2023-07-08T12:38:03.160344+0000","last_active":"2023-07-08T12:38:13.758758+0000","last_peered":"2023-07-08T12:38:13.758758+0000","last_clean":"2023-07-08T12:38:13.758758+0000","last_became_active":"2023-07-08T12:38:03.160009+0000","last_became_peered":"2023-07-08T12:38:03.160009+0000","last_unstale":"2023-07-08T12:38:13.758758+0000","last_undegraded":"2023-07-08T12:38:13.758758+0000","last_fullsized":"2023-07-08T12:38:13.758758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:35:57.492852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759529+0000","last_change":"2023-07-08T12:38:07.539629+0000","last_active":"2023-07-08T12:38:13.759529+0000","last_peered":"2023-07-08T12:38:13.759529+0000","last_clean":"2023-07-08T12:38:13.759529+0000","last_became_active":"2023-07-08T12:38:07.537615+0000","last_became_peered":"2023-07-08T12:38:07.537615+0000","last_unstale":"2023-07-08T12:38:13.759529+0000","last_undegraded":"2023-07-08T12:38:13.759529+0000","last_fullsized":"2023-07-08T12:38:13.759529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:55.151640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759477+0000","last_change":"2023-07-08T12:38:08.556275+0000","last_active":"2023-07-08T12:38:13.759477+0000","last_peered":"2023-07-08T12:38:13.759477+0000","last_clean":"2023-07-08T12:38:13.759477+0000","last_became_active":"2023-07-08T12:38:08.556016+0000","last_became_peered":"2023-07-08T12:38:08.556016+0000","last_unstale":"2023-07-08T12:38:13.759477+0000","last_undegraded":"2023-07-08T12:38:13.759477+0000","last_fullsized":"2023-07-08T12:38:13.759477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605611+0000","last_change":"2023-07-08T12:38:12.584715+0000","last_active":"2023-07-08T12:38:13.605611+0000","last_peered":"2023-07-08T12:38:13.605611+0000","last_clean":"2023-07-08T12:38:13.605611+0000","last_became_active":"2023-07-08T12:38:12.584249+0000","last_became_peered":"2023-07-08T12:38:12.584249+0000","last_unstale":"2023-07-08T12:38:13.605611+0000","last_undegraded":"2023-07-08T12:38:13.605611+0000","last_fullsized":"2023-07-08T12:38:13.605611+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:34:52.829894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759931+0000","last_change":"2023-07-08T12:38:07.537430+0000","last_active":"2023-07-08T12:38:13.759931+0000","last_peered":"2023-07-08T12:38:13.759931+0000","last_clean":"2023-07-08T12:38:13.759931+0000","last_became_active":"2023-07-08T12:38:07.537043+0000","last_became_peered":"2023-07-08T12:38:07.537043+0000","last_unstale":"2023-07-08T12:38:13.759931+0000","last_undegraded":"2023-07-08T12:38:13.759931+0000","last_fullsized":"2023-07-08T12:38:13.759931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:18:17.308237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760054+0000","last_change":"2023-07-08T12:38:02.385821+0000","last_active":"2023-07-08T12:38:13.760054+0000","last_peered":"2023-07-08T12:38:13.760054+0000","last_clean":"2023-07-08T12:38:13.760054+0000","last_became_active":"2023-07-08T12:38:02.385512+0000","last_became_peered":"2023-07-08T12:38:02.385512+0000","last_unstale":"2023-07-08T12:38:13.760054+0000","last_undegraded":"2023-07-08T12:38:13.760054+0000","last_fullsized":"2023-07-08T12:38:13.760054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:49.796115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758149+0000","last_change":"2023-07-08T12:38:08.556555+0000","last_active":"2023-07-08T12:38:13.758149+0000","last_peered":"2023-07-08T12:38:13.758149+0000","last_clean":"2023-07-08T12:38:13.758149+0000","last_became_active":"2023-07-08T12:38:08.555822+0000","last_became_peered":"2023-07-08T12:38:08.555822+0000","last_unstale":"2023-07-08T12:38:13.758149+0000","last_undegraded":"2023-07-08T12:38:13.758149+0000","last_fullsized":"2023-07-08T12:38:13.758149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:15.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758115+0000","last_change":"2023-07-08T12:38:12.587833+0000","last_active":"2023-07-08T12:38:13.758115+0000","last_peered":"2023-07-08T12:38:13.758115+0000","last_clean":"2023-07-08T12:38:13.758115+0000","last_became_active":"2023-07-08T12:38:12.587465+0000","last_became_peered":"2023-07-08T12:38:12.587465+0000","last_unstale":"2023-07-08T12:38:13.758115+0000","last_undegraded":"2023-07-08T12:38:13.758115+0000","last_fullsized":"2023-07-08T12:38:13.758115+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:32.048081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759111+0000","last_change":"2023-07-08T12:38:02.385388+0000","last_active":"2023-07-08T12:38:13.759111+0000","last_peered":"2023-07-08T12:38:13.759111+0000","last_clean":"2023-07-08T12:38:13.759111+0000","last_became_active":"2023-07-08T12:38:02.385110+0000","last_became_peered":"2023-07-08T12:38:02.385110+0000","last_unstale":"2023-07-08T12:38:13.759111+0000","last_undegraded":"2023-07-08T12:38:13.759111+0000","last_fullsized":"2023-07-08T12:38:13.759111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:52:02.887135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759215+0000","last_change":"2023-07-08T12:38:07.544210+0000","last_active":"2023-07-08T12:38:13.759215+0000","last_peered":"2023-07-08T12:38:13.759215+0000","last_clean":"2023-07-08T12:38:13.759215+0000","last_became_active":"2023-07-08T12:38:07.543870+0000","last_became_peered":"2023-07-08T12:38:07.543870+0000","last_unstale":"2023-07-08T12:38:13.759215+0000","last_undegraded":"2023-07-08T12:38:13.759215+0000","last_fullsized":"2023-07-08T12:38:13.759215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:27.000382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606243+0000","last_change":"2023-07-08T12:38:08.548933+0000","last_active":"2023-07-08T12:38:13.606243+0000","last_peered":"2023-07-08T12:38:13.606243+0000","last_clean":"2023-07-08T12:38:13.606243+0000","last_became_active":"2023-07-08T12:38:08.548655+0000","last_became_peered":"2023-07-08T12:38:08.548655+0000","last_unstale":"2023-07-08T12:38:13.606243+0000","last_undegraded":"2023-07-08T12:38:13.606243+0000","last_fullsized":"2023-07-08T12:38:13.606243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:33:14.696652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759816+0000","last_change":"2023-07-08T12:38:12.597637+0000","last_active":"2023-07-08T12:38:13.759816+0000","last_peered":"2023-07-08T12:38:13.759816+0000","last_clean":"2023-07-08T12:38:13.759816+0000","last_became_active":"2023-07-08T12:38:12.597184+0000","last_became_peered":"2023-07-08T12:38:12.597184+0000","last_unstale":"2023-07-08T12:38:13.759816+0000","last_undegraded":"2023-07-08T12:38:13.759816+0000","last_fullsized":"2023-07-08T12:38:13.759816+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:44.677613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759341+0000","last_change":"2023-07-08T12:38:07.546269+0000","last_active":"2023-07-08T12:38:13.759341+0000","last_peered":"2023-07-08T12:38:13.759341+0000","last_clean":"2023-07-08T12:38:13.759341+0000","last_became_active":"2023-07-08T12:38:07.545942+0000","last_became_peered":"2023-07-08T12:38:07.545942+0000","last_unstale":"2023-07-08T12:38:13.759341+0000","last_undegraded":"2023-07-08T12:38:13.759341+0000","last_fullsized":"2023-07-08T12:38:13.759341+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:55.151640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604575+0000","last_change":"2023-07-08T12:38:08.566129+0000","last_active":"2023-07-08T12:38:13.604575+0000","last_peered":"2023-07-08T12:38:13.604575+0000","last_clean":"2023-07-08T12:38:13.604575+0000","last_became_active":"2023-07-08T12:38:08.565780+0000","last_became_peered":"2023-07-08T12:38:08.565780+0000","last_unstale":"2023-07-08T12:38:13.604575+0000","last_undegraded":"2023-07-08T12:38:13.604575+0000","last_fullsized":"2023-07-08T12:38:13.604575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:52:43.348476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758357+0000","last_change":"2023-07-08T12:38:12.582590+0000","last_active":"2023-07-08T12:38:13.758357+0000","last_peered":"2023-07-08T12:38:13.758357+0000","last_clean":"2023-07-08T12:38:13.758357+0000","last_became_active":"2023-07-08T12:38:12.582179+0000","last_became_peered":"2023-07-08T12:38:12.582179+0000","last_unstale":"2023-07-08T12:38:13.758357+0000","last_undegraded":"2023-07-08T12:38:13.758357+0000","last_fullsized":"2023-07-08T12:38:13.758357+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:39.801225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604721+0000","last_change":"2023-07-08T12:38:07.544462+0000","last_active":"2023-07-08T12:38:13.604721+0000","last_peered":"2023-07-08T12:38:13.604721+0000","last_clean":"2023-07-08T12:38:13.604721+0000","last_became_active":"2023-07-08T12:38:07.544171+0000","last_became_peered":"2023-07-08T12:38:07.544171+0000","last_unstale":"2023-07-08T12:38:13.604721+0000","last_undegraded":"2023-07-08T12:38:13.604721+0000","last_fullsized":"2023-07-08T12:38:13.604721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:14.285892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605091+0000","last_change":"2023-07-08T12:38:08.557546+0000","last_active":"2023-07-08T12:38:13.605091+0000","last_peered":"2023-07-08T12:38:13.605091+0000","last_clean":"2023-07-08T12:38:13.605091+0000","last_became_active":"2023-07-08T12:38:08.557262+0000","last_became_peered":"2023-07-08T12:38:08.557262+0000","last_unstale":"2023-07-08T12:38:13.605091+0000","last_undegraded":"2023-07-08T12:38:13.605091+0000","last_fullsized":"2023-07-08T12:38:13.605091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:11:12.440266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605361+0000","last_change":"2023-07-08T12:38:12.597319+0000","last_active":"2023-07-08T12:38:13.605361+0000","last_peered":"2023-07-08T12:38:13.605361+0000","last_clean":"2023-07-08T12:38:13.605361+0000","last_became_active":"2023-07-08T12:38:12.596928+0000","last_became_peered":"2023-07-08T12:38:12.596928+0000","last_unstale":"2023-07-08T12:38:13.605361+0000","last_undegraded":"2023-07-08T12:38:13.605361+0000","last_fullsized":"2023-07-08T12:38:13.605361+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:39.375709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604658+0000","last_change":"2023-07-08T12:38:07.540676+0000","last_active":"2023-07-08T12:38:13.604658+0000","last_peered":"2023-07-08T12:38:13.604658+0000","last_clean":"2023-07-08T12:38:13.604658+0000","last_became_active":"2023-07-08T12:38:07.540389+0000","last_became_peered":"2023-07-08T12:38:07.540389+0000","last_unstale":"2023-07-08T12:38:13.604658+0000","last_undegraded":"2023-07-08T12:38:13.604658+0000","last_fullsized":"2023-07-08T12:38:13.604658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:27.000382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606089+0000","last_change":"2023-07-08T12:38:08.546186+0000","last_active":"2023-07-08T12:38:13.606089+0000","last_peered":"2023-07-08T12:38:13.606089+0000","last_clean":"2023-07-08T12:38:13.606089+0000","last_became_active":"2023-07-08T12:38:08.545902+0000","last_became_peered":"2023-07-08T12:38:08.545902+0000","last_unstale":"2023-07-08T12:38:13.606089+0000","last_undegraded":"2023-07-08T12:38:13.606089+0000","last_fullsized":"2023-07-08T12:38:13.606089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:37:20.499952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604484+0000","last_change":"2023-07-08T12:38:12.581579+0000","last_active":"2023-07-08T12:38:13.604484+0000","last_peered":"2023-07-08T12:38:13.604484+0000","last_clean":"2023-07-08T12:38:13.604484+0000","last_became_active":"2023-07-08T12:38:12.581118+0000","last_became_peered":"2023-07-08T12:38:12.581118+0000","last_unstale":"2023-07-08T12:38:13.604484+0000","last_undegraded":"2023-07-08T12:38:13.604484+0000","last_fullsized":"2023-07-08T12:38:13.604484+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:48:23.748317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758339+0000","last_change":"2023-07-08T12:38:07.541066+0000","last_active":"2023-07-08T12:38:13.758339+0000","last_peered":"2023-07-08T12:38:13.758339+0000","last_clean":"2023-07-08T12:38:13.758339+0000","last_became_active":"2023-07-08T12:38:07.540873+0000","last_became_peered":"2023-07-08T12:38:07.540873+0000","last_unstale":"2023-07-08T12:38:13.758339+0000","last_undegraded":"2023-07-08T12:38:13.758339+0000","last_fullsized":"2023-07-08T12:38:13.758339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:34.328010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758518+0000","last_change":"2023-07-08T12:38:08.553600+0000","last_active":"2023-07-08T12:38:13.758518+0000","last_peered":"2023-07-08T12:38:13.758518+0000","last_clean":"2023-07-08T12:38:13.758518+0000","last_became_active":"2023-07-08T12:38:08.552418+0000","last_became_peered":"2023-07-08T12:38:08.552418+0000","last_unstale":"2023-07-08T12:38:13.758518+0000","last_undegraded":"2023-07-08T12:38:13.758518+0000","last_fullsized":"2023-07-08T12:38:13.758518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:36:03.651816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758948+0000","last_change":"2023-07-08T12:38:12.590123+0000","last_active":"2023-07-08T12:38:13.758948+0000","last_peered":"2023-07-08T12:38:13.758948+0000","last_clean":"2023-07-08T12:38:13.758948+0000","last_became_active":"2023-07-08T12:38:12.589586+0000","last_became_peered":"2023-07-08T12:38:12.589586+0000","last_unstale":"2023-07-08T12:38:13.758948+0000","last_undegraded":"2023-07-08T12:38:13.758948+0000","last_fullsized":"2023-07-08T12:38:13.758948+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:03:21.918481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759739+0000","last_change":"2023-07-08T12:38:07.541648+0000","last_active":"2023-07-08T12:38:13.759739+0000","last_peered":"2023-07-08T12:38:13.759739+0000","last_clean":"2023-07-08T12:38:13.759739+0000","last_became_active":"2023-07-08T12:38:07.541284+0000","last_became_peered":"2023-07-08T12:38:07.541284+0000","last_unstale":"2023-07-08T12:38:13.759739+0000","last_undegraded":"2023-07-08T12:38:13.759739+0000","last_fullsized":"2023-07-08T12:38:13.759739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:36.285500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606128+0000","last_change":"2023-07-08T12:38:08.552907+0000","last_active":"2023-07-08T12:38:13.606128+0000","last_peered":"2023-07-08T12:38:13.606128+0000","last_clean":"2023-07-08T12:38:13.606128+0000","last_became_active":"2023-07-08T12:38:08.552595+0000","last_became_peered":"2023-07-08T12:38:08.552595+0000","last_unstale":"2023-07-08T12:38:13.606128+0000","last_undegraded":"2023-07-08T12:38:13.606128+0000","last_fullsized":"2023-07-08T12:38:13.606128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:02:09.276408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759378+0000","last_change":"2023-07-08T12:38:08.554379+0000","last_active":"2023-07-08T12:38:13.759378+0000","last_peered":"2023-07-08T12:38:13.759378+0000","last_clean":"2023-07-08T12:38:13.759378+0000","last_became_active":"2023-07-08T12:38:08.554092+0000","last_became_peered":"2023-07-08T12:38:08.554092+0000","last_unstale":"2023-07-08T12:38:13.759378+0000","last_undegraded":"2023-07-08T12:38:13.759378+0000","last_fullsized":"2023-07-08T12:38:13.759378+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:36:56.254060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758785+0000","last_change":"2023-07-08T12:38:08.557447+0000","last_active":"2023-07-08T12:38:13.758785+0000","last_peered":"2023-07-08T12:38:13.758785+0000","last_clean":"2023-07-08T12:38:13.758785+0000","last_became_active":"2023-07-08T12:38:08.557112+0000","last_became_peered":"2023-07-08T12:38:08.557112+0000","last_unstale":"2023-07-08T12:38:13.758785+0000","last_undegraded":"2023-07-08T12:38:13.758785+0000","last_fullsized":"2023-07-08T12:38:13.758785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:31:53.990746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606414+0000","last_change":"2023-07-08T12:38:08.550996+0000","last_active":"2023-07-08T12:38:13.606414+0000","last_peered":"2023-07-08T12:38:13.606414+0000","last_clean":"2023-07-08T12:38:13.606414+0000","last_became_active":"2023-07-08T12:38:08.550597+0000","last_became_peered":"2023-07-08T12:38:08.550597+0000","last_unstale":"2023-07-08T12:38:13.606414+0000","last_undegraded":"2023-07-08T12:38:13.606414+0000","last_fullsized":"2023-07-08T12:38:13.606414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:44:57.917258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758464+0000","last_change":"2023-07-08T12:38:08.552118+0000","last_active":"2023-07-08T12:38:13.758464+0000","last_peered":"2023-07-08T12:38:13.758464+0000","last_clean":"2023-07-08T12:38:13.758464+0000","last_became_active":"2023-07-08T12:38:08.551258+0000","last_became_peered":"2023-07-08T12:38:08.551258+0000","last_unstale":"2023-07-08T12:38:13.758464+0000","last_undegraded":"2023-07-08T12:38:13.758464+0000","last_fullsized":"2023-07-08T12:38:13.758464+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:22.667819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759628+0000","last_change":"2023-07-08T12:38:08.563050+0000","last_active":"2023-07-08T12:38:13.759628+0000","last_peered":"2023-07-08T12:38:13.759628+0000","last_clean":"2023-07-08T12:38:13.759628+0000","last_became_active":"2023-07-08T12:38:08.562779+0000","last_became_peered":"2023-07-08T12:38:08.562779+0000","last_unstale":"2023-07-08T12:38:13.759628+0000","last_undegraded":"2023-07-08T12:38:13.759628+0000","last_fullsized":"2023-07-08T12:38:13.759628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:15.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4592,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":718950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":901120,"data_stored":715522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":323584,"data_stored":128422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":311296,"data_stored":125224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-08T12:38:38.282 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-08T12:38:38.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.438+0000 7f0c6a6af700 1 Processor -- start 2023-07-08T12:38:38.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.438+0000 7f0c6a6af700 1 -- start start 2023-07-08T12:38:38.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c6a6af700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0c6415a480 0x7f0c640cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c6a6af700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c640b2410 0x7f0c640b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c6a6af700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6415a850 con 0x7f0c6415def0 2023-07-08T12:38:38.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c6a6af700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c640b2da0 con 0x7f0c6415a480 2023-07-08T12:38:38.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c6a6af700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c640cde90 con 0x7f0c640b2410 2023-07-08T12:38:38.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c63fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0c6415a480 0x7f0c640cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c63fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0c6415a480 0x7f0c640cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50010/0 (socket says 172.21.15.12:50010) 2023-07-08T12:38:38.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c63fff700 1 -- 172.21.15.12:0/3639540277 learned_addr learned my addr 172.21.15.12:0/3639540277 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:38.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c637fe700 1 --2- 172.21.15.12:0/3639540277 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c640b2410 0x7f0c640b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c63fff700 1 -- 172.21.15.12:0/3639540277 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c640b2410 msgr2=0x7f0c640b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:38.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c63fff700 1 --2- 172.21.15.12:0/3639540277 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c640b2410 0x7f0c640b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c63fff700 1 -- 172.21.15.12:0/3639540277 >> v1:172.21.15.160:6789/0 conn(0x7f0c6415def0 legacy=0x7f0c6415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:38.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c63fff700 1 -- 172.21.15.12:0/3639540277 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c640cd8d0 con 0x7f0c6415a480 2023-07-08T12:38:38.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.439+0000 7f0c637fe700 1 --2- 172.21.15.12:0/3639540277 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c640b2410 0x7f0c640b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:38.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.440+0000 7f0c63fff700 1 --2- 172.21.15.12:0/3639540277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c6415a480 0x7f0c640cafe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0c580098f0 tx=0x7f0c58009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195e282d83c74153 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:38.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.440+0000 7f0c62ffd700 1 -- 172.21.15.12:0/3639540277 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58004230 con 0x7f0c6415a480 2023-07-08T12:38:38.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.440+0000 7f0c62ffd700 1 -- 172.21.15.12:0/3639540277 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c58004390 con 0x7f0c6415a480 2023-07-08T12:38:38.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.440+0000 7f0c62ffd700 1 -- 172.21.15.12:0/3639540277 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c580036a0 con 0x7f0c6415a480 2023-07-08T12:38:38.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.440+0000 7f0c6a6af700 1 -- 172.21.15.12:0/3639540277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c6415a480 msgr2=0x7f0c640cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.440+0000 7f0c6a6af700 1 --2- 172.21.15.12:0/3639540277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c6415a480 0x7f0c640cafe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0c580098f0 tx=0x7f0c58009c00 comp rx=0 tx=0).stop 2023-07-08T12:38:38.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.440+0000 7f0c6a6af700 1 -- 172.21.15.12:0/3639540277 shutdown_connections 2023-07-08T12:38:38.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.440+0000 7f0c6a6af700 1 --2- 172.21.15.12:0/3639540277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c6415a480 0x7f0c640cafe0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.440+0000 7f0c6a6af700 1 --2- 172.21.15.12:0/3639540277 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c640b2410 0x7f0c640b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.440+0000 7f0c6a6af700 1 -- 172.21.15.12:0/3639540277 >> 172.21.15.12:0/3639540277 conn(0x7f0c6406ab60 msgr2=0x7f0c6406e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:38.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.440+0000 7f0c6a6af700 1 -- 172.21.15.12:0/3639540277 shutdown_connections 2023-07-08T12:38:38.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.440+0000 7f0c6a6af700 1 -- 172.21.15.12:0/3639540277 wait complete. 2023-07-08T12:38:38.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.441+0000 7f0c6a6af700 1 Processor -- start 2023-07-08T12:38:38.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.442+0000 7f0c6a6af700 1 -- start start 2023-07-08T12:38:38.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.442+0000 7f0c6a6af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c640b2410 0x7f0c641f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.442+0000 7f0c6a6af700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c6415a480 0x7f0c640b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.442+0000 7f0c6a6af700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c6415def0 0x7f0c6406b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.442+0000 7f0c6a6af700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c64151670 con 0x7f0c640b2410 2023-07-08T12:38:38.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.442+0000 7f0c6a6af700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c640cffc0 con 0x7f0c6415a480 2023-07-08T12:38:38.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.442+0000 7f0c6a6af700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c641f66f0 con 0x7f0c6415def0 2023-07-08T12:38:38.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c63fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c640b2410 0x7f0c641f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c63fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c640b2410 0x7f0c641f57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50016/0 (socket says 172.21.15.12:50016) 2023-07-08T12:38:38.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c63fff700 1 -- 172.21.15.12:0/1239359234 learned_addr learned my addr 172.21.15.12:0/1239359234 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:38.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c63fff700 1 -- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c6415def0 msgr2=0x7f0c6406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c637fe700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c6415a480 0x7f0c640b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c68c4c700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c6415def0 0x7f0c6406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c63fff700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c6415def0 0x7f0c6406b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c63fff700 1 -- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c6415a480 msgr2=0x7f0c640b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c63fff700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c6415a480 0x7f0c640b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c63fff700 1 -- 172.21.15.12:0/1239359234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c58009550 con 0x7f0c640b2410 2023-07-08T12:38:38.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c637fe700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c6415a480 0x7f0c640b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:38.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c68c4c700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c6415def0 0x7f0c6406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:38.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c63fff700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c640b2410 0x7f0c641f57c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009130 tx=0x7f0c58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:38.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c60ff9700 1 -- 172.21.15.12:0/1239359234 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5800b660 con 0x7f0c640b2410 2023-07-08T12:38:38.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c60ff9700 1 -- 172.21.15.12:0/1239359234 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5800b7c0 con 0x7f0c640b2410 2023-07-08T12:38:38.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c6a6af700 1 -- 172.21.15.12:0/1239359234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c641f5e50 con 0x7f0c640b2410 2023-07-08T12:38:38.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c60ff9700 1 -- 172.21.15.12:0/1239359234 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5800bb70 con 0x7f0c640b2410 2023-07-08T12:38:38.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.443+0000 7f0c6a6af700 1 -- 172.21.15.12:0/1239359234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c641f6410 con 0x7f0c640b2410 2023-07-08T12:38:38.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.444+0000 7f0c6a6af700 1 -- 172.21.15.12:0/1239359234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6404ea60 con 0x7f0c640b2410 2023-07-08T12:38:38.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.445+0000 7f0c60ff9700 1 -- 172.21.15.12:0/1239359234 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0c5800bcd0 con 0x7f0c640b2410 2023-07-08T12:38:38.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.445+0000 7f0c60ff9700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0c5006ace0 0x7f0c5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.445+0000 7f0c60ff9700 1 -- 172.21.15.12:0/1239359234 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0c5808ad90 con 0x7f0c640b2410 2023-07-08T12:38:38.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.445+0000 7f0c637fe700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0c5006ace0 0x7f0c5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.446+0000 7f0c637fe700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0c5006ace0 0x7f0c5006d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0c54008910 tx=0x7f0c54008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:38.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.450+0000 7f0c60ff9700 1 -- 172.21.15.12:0/1239359234 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0c58055be0 con 0x7f0c640b2410 2023-07-08T12:38:38.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.645+0000 7f0c6a6af700 1 -- 172.21.15.12:0/1239359234 --> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c640ce690 con 0x7f0c5006ace0 2023-07-08T12:38:38.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.655+0000 7f0c60ff9700 1 -- 172.21.15.12:0/1239359234 <== mgr.4103 v2:172.21.15.160:6832/550875184 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367347 (secure 0 0 0) 0x7f0c5400c040 con 0x7f0c5006ace0 2023-07-08T12:38:38.663 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:38.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.659+0000 7f0c6a6af700 1 -- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0c5006ace0 msgr2=0x7f0c5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.659+0000 7f0c6a6af700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0c5006ace0 0x7f0c5006d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0c54008910 tx=0x7f0c54008040 comp rx=0 tx=0).stop 2023-07-08T12:38:38.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.659+0000 7f0c6a6af700 1 -- 172.21.15.12:0/1239359234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c640b2410 msgr2=0x7f0c641f57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.659+0000 7f0c6a6af700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c640b2410 0x7f0c641f57c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009130 tx=0x7f0c58004750 comp rx=0 tx=0).stop 2023-07-08T12:38:38.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.659+0000 7f0c6a6af700 1 -- 172.21.15.12:0/1239359234 shutdown_connections 2023-07-08T12:38:38.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.659+0000 7f0c6a6af700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0c5006ace0 0x7f0c5006d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.659+0000 7f0c6a6af700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c6415def0 0x7f0c6406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.659+0000 7f0c6a6af700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c640b2410 0x7f0c641f57c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.659+0000 7f0c6a6af700 1 --2- 172.21.15.12:0/1239359234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c6415a480 0x7f0c640b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.659+0000 7f0c6a6af700 1 -- 172.21.15.12:0/1239359234 >> 172.21.15.12:0/1239359234 conn(0x7f0c6406ab60 msgr2=0x7f0c640cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:38.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.659+0000 7f0c6a6af700 1 -- 172.21.15.12:0/1239359234 shutdown_connections 2023-07-08T12:38:38.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.659+0000 7f0c6a6af700 1 -- 172.21.15.12:0/1239359234 wait complete. 2023-07-08T12:38:38.669 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-07-08T12:38:38.672 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2023-07-08T12:38:38.147687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594960,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219208,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755512,"statfs":{"total":773094113280,"available":772869644288,"internally_reserved":0,"allocated":3678208,"data_stored":2186496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003828"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604481+0000","last_change":"2023-07-08T12:38:08.558253+0000","last_active":"2023-07-08T12:38:13.604481+0000","last_peered":"2023-07-08T12:38:13.604481+0000","last_clean":"2023-07-08T12:38:13.604481+0000","last_became_active":"2023-07-08T12:38:08.557970+0000","last_became_peered":"2023-07-08T12:38:08.557970+0000","last_unstale":"2023-07-08T12:38:13.604481+0000","last_undegraded":"2023-07-08T12:38:13.604481+0000","last_fullsized":"2023-07-08T12:38:13.604481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:56:17.613729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.608666+0000","last_change":"2023-07-08T12:38:08.546715+0000","last_active":"2023-07-08T12:38:13.608666+0000","last_peered":"2023-07-08T12:38:13.608666+0000","last_clean":"2023-07-08T12:38:13.608666+0000","last_became_active":"2023-07-08T12:38:08.546365+0000","last_became_peered":"2023-07-08T12:38:08.546365+0000","last_unstale":"2023-07-08T12:38:13.608666+0000","last_undegraded":"2023-07-08T12:38:13.608666+0000","last_fullsized":"2023-07-08T12:38:13.608666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:55:50.492319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606777+0000","last_change":"2023-07-08T12:38:08.561903+0000","last_active":"2023-07-08T12:38:13.606777+0000","last_peered":"2023-07-08T12:38:13.606777+0000","last_clean":"2023-07-08T12:38:13.606777+0000","last_became_active":"2023-07-08T12:38:08.561533+0000","last_became_peered":"2023-07-08T12:38:08.561533+0000","last_unstale":"2023-07-08T12:38:13.606777+0000","last_undegraded":"2023-07-08T12:38:13.606777+0000","last_fullsized":"2023-07-08T12:38:13.606777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:26:30.519413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758858+0000","last_change":"2023-07-08T12:38:08.556755+0000","last_active":"2023-07-08T12:38:13.758858+0000","last_peered":"2023-07-08T12:38:13.758858+0000","last_clean":"2023-07-08T12:38:13.758858+0000","last_became_active":"2023-07-08T12:38:08.556476+0000","last_became_peered":"2023-07-08T12:38:08.556476+0000","last_unstale":"2023-07-08T12:38:13.758858+0000","last_undegraded":"2023-07-08T12:38:13.758858+0000","last_fullsized":"2023-07-08T12:38:13.758858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:52.854234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759575+0000","last_change":"2023-07-08T12:38:08.561313+0000","last_active":"2023-07-08T12:38:13.759575+0000","last_peered":"2023-07-08T12:38:13.759575+0000","last_clean":"2023-07-08T12:38:13.759575+0000","last_became_active":"2023-07-08T12:38:08.561055+0000","last_became_peered":"2023-07-08T12:38:08.561055+0000","last_unstale":"2023-07-08T12:38:13.759575+0000","last_undegraded":"2023-07-08T12:38:13.759575+0000","last_fullsized":"2023-07-08T12:38:13.759575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:49.502059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606630+0000","last_change":"2023-07-08T12:38:08.564651+0000","last_active":"2023-07-08T12:38:13.606630+0000","last_peered":"2023-07-08T12:38:13.606630+0000","last_clean":"2023-07-08T12:38:13.606630+0000","last_became_active":"2023-07-08T12:38:08.564389+0000","last_became_peered":"2023-07-08T12:38:08.564389+0000","last_unstale":"2023-07-08T12:38:13.606630+0000","last_undegraded":"2023-07-08T12:38:13.606630+0000","last_fullsized":"2023-07-08T12:38:13.606630+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:03:39.903947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758619+0000","last_change":"2023-07-08T12:38:08.569425+0000","last_active":"2023-07-08T12:38:13.758619+0000","last_peered":"2023-07-08T12:38:13.758619+0000","last_clean":"2023-07-08T12:38:13.758619+0000","last_became_active":"2023-07-08T12:38:08.569167+0000","last_became_peered":"2023-07-08T12:38:08.569167+0000","last_unstale":"2023-07-08T12:38:13.758619+0000","last_undegraded":"2023-07-08T12:38:13.758619+0000","last_fullsized":"2023-07-08T12:38:13.758619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:07.081181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605308+0000","last_change":"2023-07-08T12:38:08.549000+0000","last_active":"2023-07-08T12:38:13.605308+0000","last_peered":"2023-07-08T12:38:13.605308+0000","last_clean":"2023-07-08T12:38:13.605308+0000","last_became_active":"2023-07-08T12:38:08.548657+0000","last_became_peered":"2023-07-08T12:38:08.548657+0000","last_unstale":"2023-07-08T12:38:13.605308+0000","last_undegraded":"2023-07-08T12:38:13.605308+0000","last_fullsized":"2023-07-08T12:38:13.605308+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:37.290100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759490+0000","last_change":"2023-07-08T12:38:08.565002+0000","last_active":"2023-07-08T12:38:13.759490+0000","last_peered":"2023-07-08T12:38:13.759490+0000","last_clean":"2023-07-08T12:38:13.759490+0000","last_became_active":"2023-07-08T12:38:08.564554+0000","last_became_peered":"2023-07-08T12:38:08.564554+0000","last_unstale":"2023-07-08T12:38:13.759490+0000","last_undegraded":"2023-07-08T12:38:13.759490+0000","last_fullsized":"2023-07-08T12:38:13.759490+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:35.758126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606670+0000","last_change":"2023-07-08T12:38:08.556118+0000","last_active":"2023-07-08T12:38:13.606670+0000","last_peered":"2023-07-08T12:38:13.606670+0000","last_clean":"2023-07-08T12:38:13.606670+0000","last_became_active":"2023-07-08T12:38:08.555833+0000","last_became_peered":"2023-07-08T12:38:08.555833+0000","last_unstale":"2023-07-08T12:38:13.606670+0000","last_undegraded":"2023-07-08T12:38:13.606670+0000","last_fullsized":"2023-07-08T12:38:13.606670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:03.390028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606704+0000","last_change":"2023-07-08T12:38:08.568872+0000","last_active":"2023-07-08T12:38:13.606704+0000","last_peered":"2023-07-08T12:38:13.606704+0000","last_clean":"2023-07-08T12:38:13.606704+0000","last_became_active":"2023-07-08T12:38:08.568624+0000","last_became_peered":"2023-07-08T12:38:08.568624+0000","last_unstale":"2023-07-08T12:38:13.606704+0000","last_undegraded":"2023-07-08T12:38:13.606704+0000","last_fullsized":"2023-07-08T12:38:13.606704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:35:09.678657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604156+0000","last_change":"2023-07-08T12:38:08.560950+0000","last_active":"2023-07-08T12:38:13.604156+0000","last_peered":"2023-07-08T12:38:13.604156+0000","last_clean":"2023-07-08T12:38:13.604156+0000","last_became_active":"2023-07-08T12:38:08.560621+0000","last_became_peered":"2023-07-08T12:38:08.560621+0000","last_unstale":"2023-07-08T12:38:13.604156+0000","last_undegraded":"2023-07-08T12:38:13.604156+0000","last_fullsized":"2023-07-08T12:38:13.604156+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T12:59:32.693976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604193+0000","last_change":"2023-07-08T12:38:08.556685+0000","last_active":"2023-07-08T12:38:13.604193+0000","last_peered":"2023-07-08T12:38:13.604193+0000","last_clean":"2023-07-08T12:38:13.604193+0000","last_became_active":"2023-07-08T12:38:08.556387+0000","last_became_peered":"2023-07-08T12:38:08.556387+0000","last_unstale":"2023-07-08T12:38:13.604193+0000","last_undegraded":"2023-07-08T12:38:13.604193+0000","last_fullsized":"2023-07-08T12:38:13.604193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:03:36.626338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604952+0000","last_change":"2023-07-08T12:38:08.547070+0000","last_active":"2023-07-08T12:38:13.604952+0000","last_peered":"2023-07-08T12:38:13.604952+0000","last_clean":"2023-07-08T12:38:13.604952+0000","last_became_active":"2023-07-08T12:38:08.546813+0000","last_became_peered":"2023-07-08T12:38:08.546813+0000","last_unstale":"2023-07-08T12:38:13.604952+0000","last_undegraded":"2023-07-08T12:38:13.604952+0000","last_fullsized":"2023-07-08T12:38:13.604952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758137+0000","last_change":"2023-07-08T12:38:08.569414+0000","last_active":"2023-07-08T12:38:13.758137+0000","last_peered":"2023-07-08T12:38:13.758137+0000","last_clean":"2023-07-08T12:38:13.758137+0000","last_became_active":"2023-07-08T12:38:08.569177+0000","last_became_peered":"2023-07-08T12:38:08.569177+0000","last_unstale":"2023-07-08T12:38:13.758137+0000","last_undegraded":"2023-07-08T12:38:13.758137+0000","last_fullsized":"2023-07-08T12:38:13.758137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606000+0000","last_change":"2023-07-08T12:38:08.561730+0000","last_active":"2023-07-08T12:38:13.606000+0000","last_peered":"2023-07-08T12:38:13.606000+0000","last_clean":"2023-07-08T12:38:13.606000+0000","last_became_active":"2023-07-08T12:38:08.561498+0000","last_became_peered":"2023-07-08T12:38:08.561498+0000","last_unstale":"2023-07-08T12:38:13.606000+0000","last_undegraded":"2023-07-08T12:38:13.606000+0000","last_fullsized":"2023-07-08T12:38:13.606000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:06:06.921530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759585+0000","last_change":"2023-07-08T12:38:08.568400+0000","last_active":"2023-07-08T12:38:13.759585+0000","last_peered":"2023-07-08T12:38:13.759585+0000","last_clean":"2023-07-08T12:38:13.759585+0000","last_became_active":"2023-07-08T12:38:08.568070+0000","last_became_peered":"2023-07-08T12:38:08.568070+0000","last_unstale":"2023-07-08T12:38:13.759585+0000","last_undegraded":"2023-07-08T12:38:13.759585+0000","last_fullsized":"2023-07-08T12:38:13.759585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:35.332610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606886+0000","last_change":"2023-07-08T12:38:08.549069+0000","last_active":"2023-07-08T12:38:13.606886+0000","last_peered":"2023-07-08T12:38:13.606886+0000","last_clean":"2023-07-08T12:38:13.606886+0000","last_became_active":"2023-07-08T12:38:08.548774+0000","last_became_peered":"2023-07-08T12:38:08.548774+0000","last_unstale":"2023-07-08T12:38:13.606886+0000","last_undegraded":"2023-07-08T12:38:13.606886+0000","last_fullsized":"2023-07-08T12:38:13.606886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:02:05.917920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604270+0000","last_change":"2023-07-08T12:38:08.553458+0000","last_active":"2023-07-08T12:38:13.604270+0000","last_peered":"2023-07-08T12:38:13.604270+0000","last_clean":"2023-07-08T12:38:13.604270+0000","last_became_active":"2023-07-08T12:38:08.553177+0000","last_became_peered":"2023-07-08T12:38:08.553177+0000","last_unstale":"2023-07-08T12:38:13.604270+0000","last_undegraded":"2023-07-08T12:38:13.604270+0000","last_fullsized":"2023-07-08T12:38:13.604270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:07:08.413767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606438+0000","last_change":"2023-07-08T12:38:08.546039+0000","last_active":"2023-07-08T12:38:13.606438+0000","last_peered":"2023-07-08T12:38:13.606438+0000","last_clean":"2023-07-08T12:38:13.606438+0000","last_became_active":"2023-07-08T12:38:08.545723+0000","last_became_peered":"2023-07-08T12:38:08.545723+0000","last_unstale":"2023-07-08T12:38:13.606438+0000","last_undegraded":"2023-07-08T12:38:13.606438+0000","last_fullsized":"2023-07-08T12:38:13.606438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:05:03.136361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758578+0000","last_change":"2023-07-08T12:38:08.558172+0000","last_active":"2023-07-08T12:38:13.758578+0000","last_peered":"2023-07-08T12:38:13.758578+0000","last_clean":"2023-07-08T12:38:13.758578+0000","last_became_active":"2023-07-08T12:38:08.557888+0000","last_became_peered":"2023-07-08T12:38:08.557888+0000","last_unstale":"2023-07-08T12:38:13.758578+0000","last_undegraded":"2023-07-08T12:38:13.758578+0000","last_fullsized":"2023-07-08T12:38:13.758578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:37.290100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758835+0000","last_change":"2023-07-08T12:38:08.569776+0000","last_active":"2023-07-08T12:38:13.758835+0000","last_peered":"2023-07-08T12:38:13.758835+0000","last_clean":"2023-07-08T12:38:13.758835+0000","last_became_active":"2023-07-08T12:38:08.569530+0000","last_became_peered":"2023-07-08T12:38:08.569530+0000","last_unstale":"2023-07-08T12:38:13.758835+0000","last_undegraded":"2023-07-08T12:38:13.758835+0000","last_fullsized":"2023-07-08T12:38:13.758835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:18:27.333138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759672+0000","last_change":"2023-07-08T12:38:08.557720+0000","last_active":"2023-07-08T12:38:13.759672+0000","last_peered":"2023-07-08T12:38:13.759672+0000","last_clean":"2023-07-08T12:38:13.759672+0000","last_became_active":"2023-07-08T12:38:08.557390+0000","last_became_peered":"2023-07-08T12:38:08.557390+0000","last_unstale":"2023-07-08T12:38:13.759672+0000","last_undegraded":"2023-07-08T12:38:13.759672+0000","last_fullsized":"2023-07-08T12:38:13.759672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:13:13.582721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758123+0000","last_change":"2023-07-08T12:38:08.558048+0000","last_active":"2023-07-08T12:38:13.758123+0000","last_peered":"2023-07-08T12:38:13.758123+0000","last_clean":"2023-07-08T12:38:13.758123+0000","last_became_active":"2023-07-08T12:38:08.557871+0000","last_became_peered":"2023-07-08T12:38:08.557871+0000","last_unstale":"2023-07-08T12:38:13.758123+0000","last_undegraded":"2023-07-08T12:38:13.758123+0000","last_fullsized":"2023-07-08T12:38:13.758123+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:40.634514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759316+0000","last_change":"2023-07-08T12:38:08.553223+0000","last_active":"2023-07-08T12:38:13.759316+0000","last_peered":"2023-07-08T12:38:13.759316+0000","last_clean":"2023-07-08T12:38:13.759316+0000","last_became_active":"2023-07-08T12:38:08.552416+0000","last_became_peered":"2023-07-08T12:38:08.552416+0000","last_unstale":"2023-07-08T12:38:13.759316+0000","last_undegraded":"2023-07-08T12:38:13.759316+0000","last_fullsized":"2023-07-08T12:38:13.759316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:06:26.551781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760263+0000","last_change":"2023-07-08T12:38:08.553769+0000","last_active":"2023-07-08T12:38:13.760263+0000","last_peered":"2023-07-08T12:38:13.760263+0000","last_clean":"2023-07-08T12:38:13.760263+0000","last_became_active":"2023-07-08T12:38:08.553502+0000","last_became_peered":"2023-07-08T12:38:08.553502+0000","last_unstale":"2023-07-08T12:38:13.760263+0000","last_undegraded":"2023-07-08T12:38:13.760263+0000","last_fullsized":"2023-07-08T12:38:13.760263+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:35.758126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605466+0000","last_change":"2023-07-08T12:38:08.546308+0000","last_active":"2023-07-08T12:38:13.605466+0000","last_peered":"2023-07-08T12:38:13.605466+0000","last_clean":"2023-07-08T12:38:13.605466+0000","last_became_active":"2023-07-08T12:38:08.545865+0000","last_became_peered":"2023-07-08T12:38:08.545865+0000","last_unstale":"2023-07-08T12:38:13.605466+0000","last_undegraded":"2023-07-08T12:38:13.605466+0000","last_fullsized":"2023-07-08T12:38:13.605466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:22.667819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606681+0000","last_change":"2023-07-08T12:38:12.584534+0000","last_active":"2023-07-08T12:38:13.606681+0000","last_peered":"2023-07-08T12:38:13.606681+0000","last_clean":"2023-07-08T12:38:13.606681+0000","last_became_active":"2023-07-08T12:38:12.584153+0000","last_became_peered":"2023-07-08T12:38:12.584153+0000","last_unstale":"2023-07-08T12:38:13.606681+0000","last_undegraded":"2023-07-08T12:38:13.606681+0000","last_fullsized":"2023-07-08T12:38:13.606681+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:21:34.996680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606263+0000","last_change":"2023-07-08T12:38:07.547338+0000","last_active":"2023-07-08T12:38:13.606263+0000","last_peered":"2023-07-08T12:38:13.606263+0000","last_clean":"2023-07-08T12:38:13.606263+0000","last_became_active":"2023-07-08T12:38:07.547161+0000","last_became_peered":"2023-07-08T12:38:07.547161+0000","last_unstale":"2023-07-08T12:38:13.606263+0000","last_undegraded":"2023-07-08T12:38:13.606263+0000","last_fullsized":"2023-07-08T12:38:13.606263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:37:52.614496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759146+0000","last_change":"2023-07-08T12:38:08.553623+0000","last_active":"2023-07-08T12:38:13.759146+0000","last_peered":"2023-07-08T12:38:13.759146+0000","last_clean":"2023-07-08T12:38:13.759146+0000","last_became_active":"2023-07-08T12:38:08.551973+0000","last_became_peered":"2023-07-08T12:38:08.551973+0000","last_unstale":"2023-07-08T12:38:13.759146+0000","last_undegraded":"2023-07-08T12:38:13.759146+0000","last_fullsized":"2023-07-08T12:38:13.759146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:55:50.492319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759324+0000","last_change":"2023-07-08T12:38:12.590297+0000","last_active":"2023-07-08T12:38:13.759324+0000","last_peered":"2023-07-08T12:38:13.759324+0000","last_clean":"2023-07-08T12:38:13.759324+0000","last_became_active":"2023-07-08T12:38:12.589898+0000","last_became_peered":"2023-07-08T12:38:12.589898+0000","last_unstale":"2023-07-08T12:38:13.759324+0000","last_undegraded":"2023-07-08T12:38:13.759324+0000","last_fullsized":"2023-07-08T12:38:13.759324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:55.150008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758590+0000","last_change":"2023-07-08T12:38:07.548760+0000","last_active":"2023-07-08T12:38:13.758590+0000","last_peered":"2023-07-08T12:38:13.758590+0000","last_clean":"2023-07-08T12:38:13.758590+0000","last_became_active":"2023-07-08T12:38:07.548448+0000","last_became_peered":"2023-07-08T12:38:07.548448+0000","last_unstale":"2023-07-08T12:38:13.758590+0000","last_undegraded":"2023-07-08T12:38:13.758590+0000","last_fullsized":"2023-07-08T12:38:13.758590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T16:04:24.797809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759344+0000","last_change":"2023-07-08T12:38:08.561715+0000","last_active":"2023-07-08T12:38:13.759344+0000","last_peered":"2023-07-08T12:38:13.759344+0000","last_clean":"2023-07-08T12:38:13.759344+0000","last_became_active":"2023-07-08T12:38:08.561004+0000","last_became_peered":"2023-07-08T12:38:08.561004+0000","last_unstale":"2023-07-08T12:38:13.759344+0000","last_undegraded":"2023-07-08T12:38:13.759344+0000","last_fullsized":"2023-07-08T12:38:13.759344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:40.634514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759948+0000","last_change":"2023-07-08T12:38:12.593094+0000","last_active":"2023-07-08T12:38:13.759948+0000","last_peered":"2023-07-08T12:38:13.759948+0000","last_clean":"2023-07-08T12:38:13.759948+0000","last_became_active":"2023-07-08T12:38:12.592704+0000","last_became_peered":"2023-07-08T12:38:12.592704+0000","last_unstale":"2023-07-08T12:38:13.759948+0000","last_undegraded":"2023-07-08T12:38:13.759948+0000","last_fullsized":"2023-07-08T12:38:13.759948+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:26.710918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758339+0000","last_change":"2023-07-08T12:38:07.543437+0000","last_active":"2023-07-08T12:38:13.758339+0000","last_peered":"2023-07-08T12:38:13.758339+0000","last_clean":"2023-07-08T12:38:13.758339+0000","last_became_active":"2023-07-08T12:38:07.543158+0000","last_became_peered":"2023-07-08T12:38:07.543158+0000","last_unstale":"2023-07-08T12:38:13.758339+0000","last_undegraded":"2023-07-08T12:38:13.758339+0000","last_fullsized":"2023-07-08T12:38:13.758339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:48.497459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759270+0000","last_change":"2023-07-08T12:38:08.552907+0000","last_active":"2023-07-08T12:38:13.759270+0000","last_peered":"2023-07-08T12:38:13.759270+0000","last_clean":"2023-07-08T12:38:13.759270+0000","last_became_active":"2023-07-08T12:38:08.552286+0000","last_became_peered":"2023-07-08T12:38:08.552286+0000","last_unstale":"2023-07-08T12:38:13.759270+0000","last_undegraded":"2023-07-08T12:38:13.759270+0000","last_fullsized":"2023-07-08T12:38:13.759270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:11:32.912205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758741+0000","last_change":"2023-07-08T12:38:12.590030+0000","last_active":"2023-07-08T12:38:13.758741+0000","last_peered":"2023-07-08T12:38:13.758741+0000","last_clean":"2023-07-08T12:38:13.758741+0000","last_became_active":"2023-07-08T12:38:12.589580+0000","last_became_peered":"2023-07-08T12:38:12.589580+0000","last_unstale":"2023-07-08T12:38:13.758741+0000","last_undegraded":"2023-07-08T12:38:13.758741+0000","last_fullsized":"2023-07-08T12:38:13.758741+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:08:35.080577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759616+0000","last_change":"2023-07-08T12:38:07.541564+0000","last_active":"2023-07-08T12:38:13.759616+0000","last_peered":"2023-07-08T12:38:13.759616+0000","last_clean":"2023-07-08T12:38:13.759616+0000","last_became_active":"2023-07-08T12:38:07.540886+0000","last_became_peered":"2023-07-08T12:38:07.540886+0000","last_unstale":"2023-07-08T12:38:13.759616+0000","last_undegraded":"2023-07-08T12:38:13.759616+0000","last_fullsized":"2023-07-08T12:38:13.759616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:35.414408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758276+0000","last_change":"2023-07-08T12:38:08.556742+0000","last_active":"2023-07-08T12:38:13.758276+0000","last_peered":"2023-07-08T12:38:13.758276+0000","last_clean":"2023-07-08T12:38:13.758276+0000","last_became_active":"2023-07-08T12:38:08.556264+0000","last_became_peered":"2023-07-08T12:38:08.556264+0000","last_unstale":"2023-07-08T12:38:13.758276+0000","last_undegraded":"2023-07-08T12:38:13.758276+0000","last_fullsized":"2023-07-08T12:38:13.758276+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:56.156240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758009+0000","last_change":"2023-07-08T12:38:12.587899+0000","last_active":"2023-07-08T12:38:13.758009+0000","last_peered":"2023-07-08T12:38:13.758009+0000","last_clean":"2023-07-08T12:38:13.758009+0000","last_became_active":"2023-07-08T12:38:12.587591+0000","last_became_peered":"2023-07-08T12:38:12.587591+0000","last_unstale":"2023-07-08T12:38:13.758009+0000","last_undegraded":"2023-07-08T12:38:13.758009+0000","last_fullsized":"2023-07-08T12:38:13.758009+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:19.333590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605565+0000","last_change":"2023-07-08T12:38:07.536496+0000","last_active":"2023-07-08T12:38:13.605565+0000","last_peered":"2023-07-08T12:38:13.605565+0000","last_clean":"2023-07-08T12:38:13.605565+0000","last_became_active":"2023-07-08T12:38:07.535995+0000","last_became_peered":"2023-07-08T12:38:07.535995+0000","last_unstale":"2023-07-08T12:38:13.605565+0000","last_undegraded":"2023-07-08T12:38:13.605565+0000","last_fullsized":"2023-07-08T12:38:13.605565+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:28:48.361016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758283+0000","last_change":"2023-07-08T12:38:08.553479+0000","last_active":"2023-07-08T12:38:13.758283+0000","last_peered":"2023-07-08T12:38:13.758283+0000","last_clean":"2023-07-08T12:38:13.758283+0000","last_became_active":"2023-07-08T12:38:08.550664+0000","last_became_peered":"2023-07-08T12:38:08.550664+0000","last_unstale":"2023-07-08T12:38:13.758283+0000","last_undegraded":"2023-07-08T12:38:13.758283+0000","last_fullsized":"2023-07-08T12:38:13.758283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758974+0000","last_change":"2023-07-08T12:38:12.592991+0000","last_active":"2023-07-08T12:38:13.758974+0000","last_peered":"2023-07-08T12:38:13.758974+0000","last_clean":"2023-07-08T12:38:13.758974+0000","last_became_active":"2023-07-08T12:38:12.591995+0000","last_became_peered":"2023-07-08T12:38:12.591995+0000","last_unstale":"2023-07-08T12:38:13.758974+0000","last_undegraded":"2023-07-08T12:38:13.758974+0000","last_fullsized":"2023-07-08T12:38:13.758974+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:07.433127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759097+0000","last_change":"2023-07-08T12:38:07.551354+0000","last_active":"2023-07-08T12:38:13.759097+0000","last_peered":"2023-07-08T12:38:13.759097+0000","last_clean":"2023-07-08T12:38:13.759097+0000","last_became_active":"2023-07-08T12:38:07.551134+0000","last_became_peered":"2023-07-08T12:38:07.551134+0000","last_unstale":"2023-07-08T12:38:13.759097+0000","last_undegraded":"2023-07-08T12:38:13.759097+0000","last_fullsized":"2023-07-08T12:38:13.759097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:41:53.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760978+0000","last_change":"2023-07-08T12:38:08.558470+0000","last_active":"2023-07-08T12:38:13.760978+0000","last_peered":"2023-07-08T12:38:13.760978+0000","last_clean":"2023-07-08T12:38:13.760978+0000","last_became_active":"2023-07-08T12:38:08.557894+0000","last_became_peered":"2023-07-08T12:38:08.557894+0000","last_unstale":"2023-07-08T12:38:13.760978+0000","last_undegraded":"2023-07-08T12:38:13.760978+0000","last_fullsized":"2023-07-08T12:38:13.760978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:35.332610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.757971+0000","last_change":"2023-07-08T12:38:12.590703+0000","last_active":"2023-07-08T12:38:13.757971+0000","last_peered":"2023-07-08T12:38:13.757971+0000","last_clean":"2023-07-08T12:38:13.757971+0000","last_became_active":"2023-07-08T12:38:12.590268+0000","last_became_peered":"2023-07-08T12:38:12.590268+0000","last_unstale":"2023-07-08T12:38:13.757971+0000","last_undegraded":"2023-07-08T12:38:13.757971+0000","last_fullsized":"2023-07-08T12:38:13.757971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:35:36.579637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:23.855571+0000","last_change":"2023-07-08T12:38:07.544325+0000","last_active":"2023-07-08T12:38:23.855571+0000","last_peered":"2023-07-08T12:38:23.855571+0000","last_clean":"2023-07-08T12:38:23.855571+0000","last_became_active":"2023-07-08T12:38:07.543861+0000","last_became_peered":"2023-07-08T12:38:07.543861+0000","last_unstale":"2023-07-08T12:38:23.855571+0000","last_undegraded":"2023-07-08T12:38:23.855571+0000","last_fullsized":"2023-07-08T12:38:23.855571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:40:52.689398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605326+0000","last_change":"2023-07-08T12:38:08.553055+0000","last_active":"2023-07-08T12:38:13.605326+0000","last_peered":"2023-07-08T12:38:13.605326+0000","last_clean":"2023-07-08T12:38:13.605326+0000","last_became_active":"2023-07-08T12:38:08.552632+0000","last_became_peered":"2023-07-08T12:38:08.552632+0000","last_unstale":"2023-07-08T12:38:13.605326+0000","last_undegraded":"2023-07-08T12:38:13.605326+0000","last_fullsized":"2023-07-08T12:38:13.605326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:37:47.885139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605700+0000","last_change":"2023-07-08T12:38:12.598190+0000","last_active":"2023-07-08T12:38:13.605700+0000","last_peered":"2023-07-08T12:38:13.605700+0000","last_clean":"2023-07-08T12:38:13.605700+0000","last_became_active":"2023-07-08T12:38:12.597813+0000","last_became_peered":"2023-07-08T12:38:12.597813+0000","last_unstale":"2023-07-08T12:38:13.605700+0000","last_undegraded":"2023-07-08T12:38:13.605700+0000","last_fullsized":"2023-07-08T12:38:13.605700+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:41:58.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758412+0000","last_change":"2023-07-08T12:38:07.543441+0000","last_active":"2023-07-08T12:38:13.758412+0000","last_peered":"2023-07-08T12:38:13.758412+0000","last_clean":"2023-07-08T12:38:13.758412+0000","last_became_active":"2023-07-08T12:38:07.543188+0000","last_became_peered":"2023-07-08T12:38:07.543188+0000","last_unstale":"2023-07-08T12:38:13.758412+0000","last_undegraded":"2023-07-08T12:38:13.758412+0000","last_fullsized":"2023-07-08T12:38:13.758412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:34.328010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758440+0000","last_change":"2023-07-08T12:38:08.552797+0000","last_active":"2023-07-08T12:38:13.758440+0000","last_peered":"2023-07-08T12:38:13.758440+0000","last_clean":"2023-07-08T12:38:13.758440+0000","last_became_active":"2023-07-08T12:38:08.552289+0000","last_became_peered":"2023-07-08T12:38:08.552289+0000","last_unstale":"2023-07-08T12:38:13.758440+0000","last_undegraded":"2023-07-08T12:38:13.758440+0000","last_fullsized":"2023-07-08T12:38:13.758440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:07.081181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758345+0000","last_change":"2023-07-08T12:38:12.592793+0000","last_active":"2023-07-08T12:38:13.758345+0000","last_peered":"2023-07-08T12:38:13.758345+0000","last_clean":"2023-07-08T12:38:13.758345+0000","last_became_active":"2023-07-08T12:38:12.592414+0000","last_became_peered":"2023-07-08T12:38:12.592414+0000","last_unstale":"2023-07-08T12:38:13.758345+0000","last_undegraded":"2023-07-08T12:38:13.758345+0000","last_fullsized":"2023-07-08T12:38:13.758345+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:39.801225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758616+0000","last_change":"2023-07-08T12:38:07.538612+0000","last_active":"2023-07-08T12:38:13.758616+0000","last_peered":"2023-07-08T12:38:13.758616+0000","last_clean":"2023-07-08T12:38:13.758616+0000","last_became_active":"2023-07-08T12:38:07.538048+0000","last_became_peered":"2023-07-08T12:38:07.538048+0000","last_unstale":"2023-07-08T12:38:13.758616+0000","last_undegraded":"2023-07-08T12:38:13.758616+0000","last_fullsized":"2023-07-08T12:38:13.758616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:04:41.955662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605266+0000","last_change":"2023-07-08T12:38:08.546131+0000","last_active":"2023-07-08T12:38:13.605266+0000","last_peered":"2023-07-08T12:38:13.605266+0000","last_clean":"2023-07-08T12:38:13.605266+0000","last_became_active":"2023-07-08T12:38:08.545491+0000","last_became_peered":"2023-07-08T12:38:08.545491+0000","last_unstale":"2023-07-08T12:38:13.605266+0000","last_undegraded":"2023-07-08T12:38:13.605266+0000","last_fullsized":"2023-07-08T12:38:13.605266+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:36.419008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758428+0000","last_change":"2023-07-08T12:38:12.584683+0000","last_active":"2023-07-08T12:38:13.758428+0000","last_peered":"2023-07-08T12:38:13.758428+0000","last_clean":"2023-07-08T12:38:13.758428+0000","last_became_active":"2023-07-08T12:38:12.584256+0000","last_became_peered":"2023-07-08T12:38:12.584256+0000","last_unstale":"2023-07-08T12:38:13.758428+0000","last_undegraded":"2023-07-08T12:38:13.758428+0000","last_fullsized":"2023-07-08T12:38:13.758428+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:41.333199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758406+0000","last_change":"2023-07-08T12:38:07.543388+0000","last_active":"2023-07-08T12:38:13.758406+0000","last_peered":"2023-07-08T12:38:13.758406+0000","last_clean":"2023-07-08T12:38:13.758406+0000","last_became_active":"2023-07-08T12:38:07.543117+0000","last_became_peered":"2023-07-08T12:38:07.543117+0000","last_unstale":"2023-07-08T12:38:13.758406+0000","last_undegraded":"2023-07-08T12:38:13.758406+0000","last_fullsized":"2023-07-08T12:38:13.758406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:21.663219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759864+0000","last_change":"2023-07-08T12:38:08.561895+0000","last_active":"2023-07-08T12:38:13.759864+0000","last_peered":"2023-07-08T12:38:13.759864+0000","last_clean":"2023-07-08T12:38:13.759864+0000","last_became_active":"2023-07-08T12:38:08.561428+0000","last_became_peered":"2023-07-08T12:38:08.561428+0000","last_unstale":"2023-07-08T12:38:13.759864+0000","last_undegraded":"2023-07-08T12:38:13.759864+0000","last_fullsized":"2023-07-08T12:38:13.759864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:07.081181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605310+0000","last_change":"2023-07-08T12:38:12.592292+0000","last_active":"2023-07-08T12:38:13.605310+0000","last_peered":"2023-07-08T12:38:13.605310+0000","last_clean":"2023-07-08T12:38:13.605310+0000","last_became_active":"2023-07-08T12:38:12.591964+0000","last_became_peered":"2023-07-08T12:38:12.591964+0000","last_unstale":"2023-07-08T12:38:13.605310+0000","last_undegraded":"2023-07-08T12:38:13.605310+0000","last_fullsized":"2023-07-08T12:38:13.605310+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:53.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758158+0000","last_change":"2023-07-08T12:38:07.543683+0000","last_active":"2023-07-08T12:38:13.758158+0000","last_peered":"2023-07-08T12:38:13.758158+0000","last_clean":"2023-07-08T12:38:13.758158+0000","last_became_active":"2023-07-08T12:38:07.543416+0000","last_became_peered":"2023-07-08T12:38:07.543416+0000","last_unstale":"2023-07-08T12:38:13.758158+0000","last_undegraded":"2023-07-08T12:38:13.758158+0000","last_fullsized":"2023-07-08T12:38:13.758158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:56:16.609129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758366+0000","last_change":"2023-07-08T12:38:08.556977+0000","last_active":"2023-07-08T12:38:13.758366+0000","last_peered":"2023-07-08T12:38:13.758366+0000","last_clean":"2023-07-08T12:38:13.758366+0000","last_became_active":"2023-07-08T12:38:08.555976+0000","last_became_peered":"2023-07-08T12:38:08.555976+0000","last_unstale":"2023-07-08T12:38:13.758366+0000","last_undegraded":"2023-07-08T12:38:13.758366+0000","last_fullsized":"2023-07-08T12:38:13.758366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:36.419008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759039+0000","last_change":"2023-07-08T12:38:12.582526+0000","last_active":"2023-07-08T12:38:13.759039+0000","last_peered":"2023-07-08T12:38:13.759039+0000","last_clean":"2023-07-08T12:38:13.759039+0000","last_became_active":"2023-07-08T12:38:12.582130+0000","last_became_peered":"2023-07-08T12:38:12.582130+0000","last_unstale":"2023-07-08T12:38:13.759039+0000","last_undegraded":"2023-07-08T12:38:13.759039+0000","last_fullsized":"2023-07-08T12:38:13.759039+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:31:58.033845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759843+0000","last_change":"2023-07-08T12:38:07.543929+0000","last_active":"2023-07-08T12:38:13.759843+0000","last_peered":"2023-07-08T12:38:13.759843+0000","last_clean":"2023-07-08T12:38:13.759843+0000","last_became_active":"2023-07-08T12:38:07.543693+0000","last_became_peered":"2023-07-08T12:38:07.543693+0000","last_unstale":"2023-07-08T12:38:13.759843+0000","last_undegraded":"2023-07-08T12:38:13.759843+0000","last_fullsized":"2023-07-08T12:38:13.759843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:55:49.487719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604436+0000","last_change":"2023-07-08T12:38:08.548856+0000","last_active":"2023-07-08T12:38:13.604436+0000","last_peered":"2023-07-08T12:38:13.604436+0000","last_clean":"2023-07-08T12:38:13.604436+0000","last_became_active":"2023-07-08T12:38:08.548294+0000","last_became_peered":"2023-07-08T12:38:08.548294+0000","last_unstale":"2023-07-08T12:38:13.604436+0000","last_undegraded":"2023-07-08T12:38:13.604436+0000","last_fullsized":"2023-07-08T12:38:13.604436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:53.873880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.603862+0000","last_change":"2023-07-08T12:38:12.587147+0000","last_active":"2023-07-08T12:38:13.603862+0000","last_peered":"2023-07-08T12:38:13.603862+0000","last_clean":"2023-07-08T12:38:13.603862+0000","last_became_active":"2023-07-08T12:38:12.586709+0000","last_became_peered":"2023-07-08T12:38:12.586709+0000","last_unstale":"2023-07-08T12:38:13.603862+0000","last_undegraded":"2023-07-08T12:38:13.603862+0000","last_fullsized":"2023-07-08T12:38:13.603862+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T12:58:56.632605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759614+0000","last_change":"2023-07-08T12:38:07.545521+0000","last_active":"2023-07-08T12:38:13.759614+0000","last_peered":"2023-07-08T12:38:13.759614+0000","last_clean":"2023-07-08T12:38:13.759614+0000","last_became_active":"2023-07-08T12:38:07.545260+0000","last_became_peered":"2023-07-08T12:38:07.545260+0000","last_unstale":"2023-07-08T12:38:13.759614+0000","last_undegraded":"2023-07-08T12:38:13.759614+0000","last_fullsized":"2023-07-08T12:38:13.759614+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:39:02.385428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605419+0000","last_change":"2023-07-08T12:38:08.552222+0000","last_active":"2023-07-08T12:38:13.605419+0000","last_peered":"2023-07-08T12:38:13.605419+0000","last_clean":"2023-07-08T12:38:13.605419+0000","last_became_active":"2023-07-08T12:38:08.551940+0000","last_became_peered":"2023-07-08T12:38:08.551940+0000","last_unstale":"2023-07-08T12:38:13.605419+0000","last_undegraded":"2023-07-08T12:38:13.605419+0000","last_fullsized":"2023-07-08T12:38:13.605419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:31:34.141462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604126+0000","last_change":"2023-07-08T12:38:12.589202+0000","last_active":"2023-07-08T12:38:13.604126+0000","last_peered":"2023-07-08T12:38:13.604126+0000","last_clean":"2023-07-08T12:38:13.604126+0000","last_became_active":"2023-07-08T12:38:12.586775+0000","last_became_peered":"2023-07-08T12:38:12.586775+0000","last_unstale":"2023-07-08T12:38:13.604126+0000","last_undegraded":"2023-07-08T12:38:13.604126+0000","last_fullsized":"2023-07-08T12:38:13.604126+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:17:35.606731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758901+0000","last_change":"2023-07-08T12:38:07.542576+0000","last_active":"2023-07-08T12:38:13.758901+0000","last_peered":"2023-07-08T12:38:13.758901+0000","last_clean":"2023-07-08T12:38:13.758901+0000","last_became_active":"2023-07-08T12:38:07.542296+0000","last_became_peered":"2023-07-08T12:38:07.542296+0000","last_unstale":"2023-07-08T12:38:13.758901+0000","last_undegraded":"2023-07-08T12:38:13.758901+0000","last_fullsized":"2023-07-08T12:38:13.758901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:28:48.361016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758718+0000","last_change":"2023-07-08T12:38:08.560794+0000","last_active":"2023-07-08T12:38:13.758718+0000","last_peered":"2023-07-08T12:38:13.758718+0000","last_clean":"2023-07-08T12:38:13.758718+0000","last_became_active":"2023-07-08T12:38:08.560405+0000","last_became_peered":"2023-07-08T12:38:08.560405+0000","last_unstale":"2023-07-08T12:38:13.758718+0000","last_undegraded":"2023-07-08T12:38:13.758718+0000","last_fullsized":"2023-07-08T12:38:13.758718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:52:43.348476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604255+0000","last_change":"2023-07-08T12:38:12.585051+0000","last_active":"2023-07-08T12:38:13.604255+0000","last_peered":"2023-07-08T12:38:13.604255+0000","last_clean":"2023-07-08T12:38:13.604255+0000","last_became_active":"2023-07-08T12:38:12.583082+0000","last_became_peered":"2023-07-08T12:38:12.583082+0000","last_unstale":"2023-07-08T12:38:13.604255+0000","last_undegraded":"2023-07-08T12:38:13.604255+0000","last_fullsized":"2023-07-08T12:38:13.604255+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:02:43.298466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604866+0000","last_change":"2023-07-08T12:38:07.537900+0000","last_active":"2023-07-08T12:38:13.604866+0000","last_peered":"2023-07-08T12:38:13.604866+0000","last_clean":"2023-07-08T12:38:13.604866+0000","last_became_active":"2023-07-08T12:38:07.536896+0000","last_became_peered":"2023-07-08T12:38:07.536896+0000","last_unstale":"2023-07-08T12:38:13.604866+0000","last_undegraded":"2023-07-08T12:38:13.604866+0000","last_fullsized":"2023-07-08T12:38:13.604866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:55.151640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759269+0000","last_change":"2023-07-08T12:38:08.569854+0000","last_active":"2023-07-08T12:38:13.759269+0000","last_peered":"2023-07-08T12:38:13.759269+0000","last_clean":"2023-07-08T12:38:13.759269+0000","last_became_active":"2023-07-08T12:38:08.569549+0000","last_became_peered":"2023-07-08T12:38:08.569549+0000","last_unstale":"2023-07-08T12:38:13.759269+0000","last_undegraded":"2023-07-08T12:38:13.759269+0000","last_fullsized":"2023-07-08T12:38:13.759269+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:15.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604571+0000","last_change":"2023-07-08T12:38:12.580215+0000","last_active":"2023-07-08T12:38:13.604571+0000","last_peered":"2023-07-08T12:38:13.604571+0000","last_clean":"2023-07-08T12:38:13.604571+0000","last_became_active":"2023-07-08T12:38:12.579706+0000","last_became_peered":"2023-07-08T12:38:12.579706+0000","last_unstale":"2023-07-08T12:38:13.604571+0000","last_undegraded":"2023-07-08T12:38:13.604571+0000","last_fullsized":"2023-07-08T12:38:13.604571+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:08:35.080577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758601+0000","last_change":"2023-07-08T12:38:07.542376+0000","last_active":"2023-07-08T12:38:13.758601+0000","last_peered":"2023-07-08T12:38:13.758601+0000","last_clean":"2023-07-08T12:38:13.758601+0000","last_became_active":"2023-07-08T12:38:07.542046+0000","last_became_peered":"2023-07-08T12:38:07.542046+0000","last_unstale":"2023-07-08T12:38:13.758601+0000","last_undegraded":"2023-07-08T12:38:13.758601+0000","last_fullsized":"2023-07-08T12:38:13.758601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:28:48.361016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605011+0000","last_change":"2023-07-08T12:38:08.553132+0000","last_active":"2023-07-08T12:38:13.605011+0000","last_peered":"2023-07-08T12:38:13.605011+0000","last_clean":"2023-07-08T12:38:13.605011+0000","last_became_active":"2023-07-08T12:38:08.552711+0000","last_became_peered":"2023-07-08T12:38:08.552711+0000","last_unstale":"2023-07-08T12:38:13.605011+0000","last_undegraded":"2023-07-08T12:38:13.605011+0000","last_fullsized":"2023-07-08T12:38:13.605011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T16:24:56.036654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759123+0000","last_change":"2023-07-08T12:38:12.597633+0000","last_active":"2023-07-08T12:38:13.759123+0000","last_peered":"2023-07-08T12:38:13.759123+0000","last_clean":"2023-07-08T12:38:13.759123+0000","last_became_active":"2023-07-08T12:38:12.597211+0000","last_became_peered":"2023-07-08T12:38:12.597211+0000","last_unstale":"2023-07-08T12:38:13.759123+0000","last_undegraded":"2023-07-08T12:38:13.759123+0000","last_fullsized":"2023-07-08T12:38:13.759123+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:04:47.003361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758887+0000","last_change":"2023-07-08T12:38:07.542040+0000","last_active":"2023-07-08T12:38:13.758887+0000","last_peered":"2023-07-08T12:38:13.758887+0000","last_clean":"2023-07-08T12:38:13.758887+0000","last_became_active":"2023-07-08T12:38:07.541743+0000","last_became_peered":"2023-07-08T12:38:07.541743+0000","last_unstale":"2023-07-08T12:38:13.758887+0000","last_undegraded":"2023-07-08T12:38:13.758887+0000","last_fullsized":"2023-07-08T12:38:13.758887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:41:49.220614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606520+0000","last_change":"2023-07-08T12:38:08.548025+0000","last_active":"2023-07-08T12:38:13.606520+0000","last_peered":"2023-07-08T12:38:13.606520+0000","last_clean":"2023-07-08T12:38:13.606520+0000","last_became_active":"2023-07-08T12:38:08.547723+0000","last_became_peered":"2023-07-08T12:38:08.547723+0000","last_unstale":"2023-07-08T12:38:13.606520+0000","last_undegraded":"2023-07-08T12:38:13.606520+0000","last_fullsized":"2023-07-08T12:38:13.606520+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:01:21.861852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605658+0000","last_change":"2023-07-08T12:38:12.587540+0000","last_active":"2023-07-08T12:38:13.605658+0000","last_peered":"2023-07-08T12:38:13.605658+0000","last_clean":"2023-07-08T12:38:13.605658+0000","last_became_active":"2023-07-08T12:38:12.587180+0000","last_became_peered":"2023-07-08T12:38:12.587180+0000","last_unstale":"2023-07-08T12:38:13.605658+0000","last_undegraded":"2023-07-08T12:38:13.605658+0000","last_fullsized":"2023-07-08T12:38:13.605658+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:31:58.033845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605007+0000","last_change":"2023-07-08T12:38:07.535635+0000","last_active":"2023-07-08T12:38:13.605007+0000","last_peered":"2023-07-08T12:38:13.605007+0000","last_clean":"2023-07-08T12:38:13.605007+0000","last_became_active":"2023-07-08T12:38:07.535335+0000","last_became_peered":"2023-07-08T12:38:07.535335+0000","last_unstale":"2023-07-08T12:38:13.605007+0000","last_undegraded":"2023-07-08T12:38:13.605007+0000","last_fullsized":"2023-07-08T12:38:13.605007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:15:42.329406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606965+0000","last_change":"2023-07-08T12:38:08.553024+0000","last_active":"2023-07-08T12:38:13.606965+0000","last_peered":"2023-07-08T12:38:13.606965+0000","last_clean":"2023-07-08T12:38:13.606965+0000","last_became_active":"2023-07-08T12:38:08.552696+0000","last_became_peered":"2023-07-08T12:38:08.552696+0000","last_unstale":"2023-07-08T12:38:13.606965+0000","last_undegraded":"2023-07-08T12:38:13.606965+0000","last_fullsized":"2023-07-08T12:38:13.606965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:33:05.815083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606715+0000","last_change":"2023-07-08T12:38:12.584493+0000","last_active":"2023-07-08T12:38:13.606715+0000","last_peered":"2023-07-08T12:38:13.606715+0000","last_clean":"2023-07-08T12:38:13.606715+0000","last_became_active":"2023-07-08T12:38:12.584087+0000","last_became_peered":"2023-07-08T12:38:12.584087+0000","last_unstale":"2023-07-08T12:38:13.606715+0000","last_undegraded":"2023-07-08T12:38:13.606715+0000","last_fullsized":"2023-07-08T12:38:13.606715+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:10:54.808570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758207+0000","last_change":"2023-07-08T12:38:07.540092+0000","last_active":"2023-07-08T12:38:13.758207+0000","last_peered":"2023-07-08T12:38:13.758207+0000","last_clean":"2023-07-08T12:38:13.758207+0000","last_became_active":"2023-07-08T12:38:07.539789+0000","last_became_peered":"2023-07-08T12:38:07.539789+0000","last_unstale":"2023-07-08T12:38:13.758207+0000","last_undegraded":"2023-07-08T12:38:13.758207+0000","last_fullsized":"2023-07-08T12:38:13.758207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:39.629914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759601+0000","last_change":"2023-07-08T12:38:08.564633+0000","last_active":"2023-07-08T12:38:13.759601+0000","last_peered":"2023-07-08T12:38:13.759601+0000","last_clean":"2023-07-08T12:38:13.759601+0000","last_became_active":"2023-07-08T12:38:08.564323+0000","last_became_peered":"2023-07-08T12:38:08.564323+0000","last_unstale":"2023-07-08T12:38:13.759601+0000","last_undegraded":"2023-07-08T12:38:13.759601+0000","last_fullsized":"2023-07-08T12:38:13.759601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:28.004982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759180+0000","last_change":"2023-07-08T12:38:12.586250+0000","last_active":"2023-07-08T12:38:13.759180+0000","last_peered":"2023-07-08T12:38:13.759180+0000","last_clean":"2023-07-08T12:38:13.759180+0000","last_became_active":"2023-07-08T12:38:12.585838+0000","last_became_peered":"2023-07-08T12:38:12.585838+0000","last_unstale":"2023-07-08T12:38:13.759180+0000","last_undegraded":"2023-07-08T12:38:13.759180+0000","last_fullsized":"2023-07-08T12:38:13.759180+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:06:30.594880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759317+0000","last_change":"2023-07-08T12:38:03.160177+0000","last_active":"2023-07-08T12:38:13.759317+0000","last_peered":"2023-07-08T12:38:13.759317+0000","last_clean":"2023-07-08T12:38:13.759317+0000","last_became_active":"2023-07-08T12:38:03.159864+0000","last_became_peered":"2023-07-08T12:38:03.159864+0000","last_unstale":"2023-07-08T12:38:13.759317+0000","last_undegraded":"2023-07-08T12:38:13.759317+0000","last_fullsized":"2023-07-08T12:38:13.759317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:31:56.449203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604812+0000","last_change":"2023-07-08T12:38:07.537328+0000","last_active":"2023-07-08T12:38:13.604812+0000","last_peered":"2023-07-08T12:38:13.604812+0000","last_clean":"2023-07-08T12:38:13.604812+0000","last_became_active":"2023-07-08T12:38:07.537038+0000","last_became_peered":"2023-07-08T12:38:07.537038+0000","last_unstale":"2023-07-08T12:38:13.604812+0000","last_undegraded":"2023-07-08T12:38:13.604812+0000","last_fullsized":"2023-07-08T12:38:13.604812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:39.629914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760774+0000","last_change":"2023-07-08T12:38:08.558271+0000","last_active":"2023-07-08T12:38:13.760774+0000","last_peered":"2023-07-08T12:38:13.760774+0000","last_clean":"2023-07-08T12:38:13.760774+0000","last_became_active":"2023-07-08T12:38:08.557598+0000","last_became_peered":"2023-07-08T12:38:08.557598+0000","last_unstale":"2023-07-08T12:38:13.760774+0000","last_undegraded":"2023-07-08T12:38:13.760774+0000","last_fullsized":"2023-07-08T12:38:13.760774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:49.502059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759252+0000","last_change":"2023-07-08T12:38:12.591995+0000","last_active":"2023-07-08T12:38:13.759252+0000","last_peered":"2023-07-08T12:38:13.759252+0000","last_clean":"2023-07-08T12:38:13.759252+0000","last_became_active":"2023-07-08T12:38:12.591624+0000","last_became_peered":"2023-07-08T12:38:12.591624+0000","last_unstale":"2023-07-08T12:38:13.759252+0000","last_undegraded":"2023-07-08T12:38:13.759252+0000","last_fullsized":"2023-07-08T12:38:13.759252+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:57.916979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605001+0000","last_change":"2023-07-08T12:38:02.384639+0000","last_active":"2023-07-08T12:38:13.605001+0000","last_peered":"2023-07-08T12:38:13.605001+0000","last_clean":"2023-07-08T12:38:13.605001+0000","last_became_active":"2023-07-08T12:38:02.384372+0000","last_became_peered":"2023-07-08T12:38:02.384372+0000","last_unstale":"2023-07-08T12:38:13.605001+0000","last_undegraded":"2023-07-08T12:38:13.605001+0000","last_fullsized":"2023-07-08T12:38:13.605001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:29.599162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605155+0000","last_change":"2023-07-08T12:38:07.541393+0000","last_active":"2023-07-08T12:38:13.605155+0000","last_peered":"2023-07-08T12:38:13.605155+0000","last_clean":"2023-07-08T12:38:13.605155+0000","last_became_active":"2023-07-08T12:38:07.541155+0000","last_became_peered":"2023-07-08T12:38:07.541155+0000","last_unstale":"2023-07-08T12:38:13.605155+0000","last_undegraded":"2023-07-08T12:38:13.605155+0000","last_fullsized":"2023-07-08T12:38:13.605155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:51:13.319094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759210+0000","last_change":"2023-07-08T12:38:08.550498+0000","last_active":"2023-07-08T12:38:13.759210+0000","last_peered":"2023-07-08T12:38:13.759210+0000","last_clean":"2023-07-08T12:38:13.759210+0000","last_became_active":"2023-07-08T12:38:08.550106+0000","last_became_peered":"2023-07-08T12:38:08.550106+0000","last_unstale":"2023-07-08T12:38:13.759210+0000","last_undegraded":"2023-07-08T12:38:13.759210+0000","last_fullsized":"2023-07-08T12:38:13.759210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:18:27.333138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604527+0000","last_change":"2023-07-08T12:38:12.586451+0000","last_active":"2023-07-08T12:38:13.604527+0000","last_peered":"2023-07-08T12:38:13.604527+0000","last_clean":"2023-07-08T12:38:13.604527+0000","last_became_active":"2023-07-08T12:38:12.582823+0000","last_became_peered":"2023-07-08T12:38:12.582823+0000","last_unstale":"2023-07-08T12:38:13.604527+0000","last_undegraded":"2023-07-08T12:38:13.604527+0000","last_fullsized":"2023-07-08T12:38:13.604527+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:18:34.491455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604236+0000","last_change":"2023-07-08T12:38:03.212466+0000","last_active":"2023-07-08T12:38:13.604236+0000","last_peered":"2023-07-08T12:38:13.604236+0000","last_clean":"2023-07-08T12:38:13.604236+0000","last_became_active":"2023-07-08T12:38:03.212182+0000","last_became_peered":"2023-07-08T12:38:03.212182+0000","last_unstale":"2023-07-08T12:38:13.604236+0000","last_undegraded":"2023-07-08T12:38:13.604236+0000","last_fullsized":"2023-07-08T12:38:13.604236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:06:34.067771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758091+0000","last_change":"2023-07-08T12:38:07.538415+0000","last_active":"2023-07-08T12:38:13.758091+0000","last_peered":"2023-07-08T12:38:13.758091+0000","last_clean":"2023-07-08T12:38:13.758091+0000","last_became_active":"2023-07-08T12:38:07.537871+0000","last_became_peered":"2023-07-08T12:38:07.537871+0000","last_unstale":"2023-07-08T12:38:13.758091+0000","last_undegraded":"2023-07-08T12:38:13.758091+0000","last_fullsized":"2023-07-08T12:38:13.758091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:00:59.245798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759478+0000","last_change":"2023-07-08T12:38:08.554865+0000","last_active":"2023-07-08T12:38:13.759478+0000","last_peered":"2023-07-08T12:38:13.759478+0000","last_clean":"2023-07-08T12:38:13.759478+0000","last_became_active":"2023-07-08T12:38:08.554591+0000","last_became_peered":"2023-07-08T12:38:08.554591+0000","last_unstale":"2023-07-08T12:38:13.759478+0000","last_undegraded":"2023-07-08T12:38:13.759478+0000","last_fullsized":"2023-07-08T12:38:13.759478+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:53.873880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758470+0000","last_change":"2023-07-08T12:38:12.579919+0000","last_active":"2023-07-08T12:38:13.758470+0000","last_peered":"2023-07-08T12:38:13.758470+0000","last_clean":"2023-07-08T12:38:13.758470+0000","last_became_active":"2023-07-08T12:38:12.579500+0000","last_became_peered":"2023-07-08T12:38:12.579500+0000","last_unstale":"2023-07-08T12:38:13.758470+0000","last_undegraded":"2023-07-08T12:38:13.758470+0000","last_fullsized":"2023-07-08T12:38:13.758470+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:56:53.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605571+0000","last_change":"2023-07-08T12:38:02.380175+0000","last_active":"2023-07-08T12:38:13.605571+0000","last_peered":"2023-07-08T12:38:13.605571+0000","last_clean":"2023-07-08T12:38:13.605571+0000","last_became_active":"2023-07-08T12:38:02.379867+0000","last_became_peered":"2023-07-08T12:38:02.379867+0000","last_unstale":"2023-07-08T12:38:13.605571+0000","last_undegraded":"2023-07-08T12:38:13.605571+0000","last_fullsized":"2023-07-08T12:38:13.605571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:35:57.492852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758117+0000","last_change":"2023-07-08T12:37:58.222371+0000","last_active":"2023-07-08T12:38:13.758117+0000","last_peered":"2023-07-08T12:38:13.758117+0000","last_clean":"2023-07-08T12:38:13.758117+0000","last_became_active":"2023-07-08T12:37:58.221381+0000","last_became_peered":"2023-07-08T12:37:58.221381+0000","last_unstale":"2023-07-08T12:38:13.758117+0000","last_undegraded":"2023-07-08T12:38:13.758117+0000","last_fullsized":"2023-07-08T12:38:13.758117+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:37:56.327863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:37:56.327863+0000","last_clean_scrub_stamp":"2023-07-08T12:37:56.327863+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:44.968733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758891+0000","last_change":"2023-07-08T12:38:07.536547+0000","last_active":"2023-07-08T12:38:13.758891+0000","last_peered":"2023-07-08T12:38:13.758891+0000","last_clean":"2023-07-08T12:38:13.758891+0000","last_became_active":"2023-07-08T12:38:07.536255+0000","last_became_peered":"2023-07-08T12:38:07.536255+0000","last_unstale":"2023-07-08T12:38:13.758891+0000","last_undegraded":"2023-07-08T12:38:13.758891+0000","last_fullsized":"2023-07-08T12:38:13.758891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:41:53.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759285+0000","last_change":"2023-07-08T12:38:08.550673+0000","last_active":"2023-07-08T12:38:13.759285+0000","last_peered":"2023-07-08T12:38:13.759285+0000","last_clean":"2023-07-08T12:38:13.759285+0000","last_became_active":"2023-07-08T12:38:08.550347+0000","last_became_peered":"2023-07-08T12:38:08.550347+0000","last_unstale":"2023-07-08T12:38:13.759285+0000","last_undegraded":"2023-07-08T12:38:13.759285+0000","last_fullsized":"2023-07-08T12:38:13.759285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:03:34.056111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605442+0000","last_change":"2023-07-08T12:38:12.597251+0000","last_active":"2023-07-08T12:38:13.605442+0000","last_peered":"2023-07-08T12:38:13.605442+0000","last_clean":"2023-07-08T12:38:13.605442+0000","last_became_active":"2023-07-08T12:38:12.596856+0000","last_became_peered":"2023-07-08T12:38:12.596856+0000","last_unstale":"2023-07-08T12:38:13.605442+0000","last_undegraded":"2023-07-08T12:38:13.605442+0000","last_fullsized":"2023-07-08T12:38:13.605442+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:43:11.124280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758299+0000","last_change":"2023-07-08T12:38:02.385193+0000","last_active":"2023-07-08T12:38:13.758299+0000","last_peered":"2023-07-08T12:38:13.758299+0000","last_clean":"2023-07-08T12:38:13.758299+0000","last_became_active":"2023-07-08T12:38:02.384865+0000","last_became_peered":"2023-07-08T12:38:02.384865+0000","last_unstale":"2023-07-08T12:38:13.758299+0000","last_undegraded":"2023-07-08T12:38:13.758299+0000","last_fullsized":"2023-07-08T12:38:13.758299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:47:30.260044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.757707+0000","last_change":"2023-07-08T12:38:07.543452+0000","last_active":"2023-07-08T12:38:13.757707+0000","last_peered":"2023-07-08T12:38:13.757707+0000","last_clean":"2023-07-08T12:38:13.757707+0000","last_became_active":"2023-07-08T12:38:07.543141+0000","last_became_peered":"2023-07-08T12:38:07.543141+0000","last_unstale":"2023-07-08T12:38:13.757707+0000","last_undegraded":"2023-07-08T12:38:13.757707+0000","last_fullsized":"2023-07-08T12:38:13.757707+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:03:16.870782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759081+0000","last_change":"2023-07-08T12:38:08.570468+0000","last_active":"2023-07-08T12:38:13.759081+0000","last_peered":"2023-07-08T12:38:13.759081+0000","last_clean":"2023-07-08T12:38:13.759081+0000","last_became_active":"2023-07-08T12:38:08.570183+0000","last_became_peered":"2023-07-08T12:38:08.570183+0000","last_unstale":"2023-07-08T12:38:13.759081+0000","last_undegraded":"2023-07-08T12:38:13.759081+0000","last_fullsized":"2023-07-08T12:38:13.759081+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:32:02.608167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759172+0000","last_change":"2023-07-08T12:38:12.597186+0000","last_active":"2023-07-08T12:38:13.759172+0000","last_peered":"2023-07-08T12:38:13.759172+0000","last_clean":"2023-07-08T12:38:13.759172+0000","last_became_active":"2023-07-08T12:38:12.596772+0000","last_became_peered":"2023-07-08T12:38:12.596772+0000","last_unstale":"2023-07-08T12:38:13.759172+0000","last_undegraded":"2023-07-08T12:38:13.759172+0000","last_fullsized":"2023-07-08T12:38:13.759172+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:06:56.291254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758758+0000","last_change":"2023-07-08T12:38:03.160344+0000","last_active":"2023-07-08T12:38:13.758758+0000","last_peered":"2023-07-08T12:38:13.758758+0000","last_clean":"2023-07-08T12:38:13.758758+0000","last_became_active":"2023-07-08T12:38:03.160009+0000","last_became_peered":"2023-07-08T12:38:03.160009+0000","last_unstale":"2023-07-08T12:38:13.758758+0000","last_undegraded":"2023-07-08T12:38:13.758758+0000","last_fullsized":"2023-07-08T12:38:13.758758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:35:57.492852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759529+0000","last_change":"2023-07-08T12:38:07.539629+0000","last_active":"2023-07-08T12:38:13.759529+0000","last_peered":"2023-07-08T12:38:13.759529+0000","last_clean":"2023-07-08T12:38:13.759529+0000","last_became_active":"2023-07-08T12:38:07.537615+0000","last_became_peered":"2023-07-08T12:38:07.537615+0000","last_unstale":"2023-07-08T12:38:13.759529+0000","last_undegraded":"2023-07-08T12:38:13.759529+0000","last_fullsized":"2023-07-08T12:38:13.759529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:55.151640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759477+0000","last_change":"2023-07-08T12:38:08.556275+0000","last_active":"2023-07-08T12:38:13.759477+0000","last_peered":"2023-07-08T12:38:13.759477+0000","last_clean":"2023-07-08T12:38:13.759477+0000","last_became_active":"2023-07-08T12:38:08.556016+0000","last_became_peered":"2023-07-08T12:38:08.556016+0000","last_unstale":"2023-07-08T12:38:13.759477+0000","last_undegraded":"2023-07-08T12:38:13.759477+0000","last_fullsized":"2023-07-08T12:38:13.759477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:55.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605611+0000","last_change":"2023-07-08T12:38:12.584715+0000","last_active":"2023-07-08T12:38:13.605611+0000","last_peered":"2023-07-08T12:38:13.605611+0000","last_clean":"2023-07-08T12:38:13.605611+0000","last_became_active":"2023-07-08T12:38:12.584249+0000","last_became_peered":"2023-07-08T12:38:12.584249+0000","last_unstale":"2023-07-08T12:38:13.605611+0000","last_undegraded":"2023-07-08T12:38:13.605611+0000","last_fullsized":"2023-07-08T12:38:13.605611+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:34:52.829894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759931+0000","last_change":"2023-07-08T12:38:07.537430+0000","last_active":"2023-07-08T12:38:13.759931+0000","last_peered":"2023-07-08T12:38:13.759931+0000","last_clean":"2023-07-08T12:38:13.759931+0000","last_became_active":"2023-07-08T12:38:07.537043+0000","last_became_peered":"2023-07-08T12:38:07.537043+0000","last_unstale":"2023-07-08T12:38:13.759931+0000","last_undegraded":"2023-07-08T12:38:13.759931+0000","last_fullsized":"2023-07-08T12:38:13.759931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:18:17.308237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.760054+0000","last_change":"2023-07-08T12:38:02.385821+0000","last_active":"2023-07-08T12:38:13.760054+0000","last_peered":"2023-07-08T12:38:13.760054+0000","last_clean":"2023-07-08T12:38:13.760054+0000","last_became_active":"2023-07-08T12:38:02.385512+0000","last_became_peered":"2023-07-08T12:38:02.385512+0000","last_unstale":"2023-07-08T12:38:13.760054+0000","last_undegraded":"2023-07-08T12:38:13.760054+0000","last_fullsized":"2023-07-08T12:38:13.760054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:09:49.796115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758149+0000","last_change":"2023-07-08T12:38:08.556555+0000","last_active":"2023-07-08T12:38:13.758149+0000","last_peered":"2023-07-08T12:38:13.758149+0000","last_clean":"2023-07-08T12:38:13.758149+0000","last_became_active":"2023-07-08T12:38:08.555822+0000","last_became_peered":"2023-07-08T12:38:08.555822+0000","last_unstale":"2023-07-08T12:38:13.758149+0000","last_undegraded":"2023-07-08T12:38:13.758149+0000","last_fullsized":"2023-07-08T12:38:13.758149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:15.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758115+0000","last_change":"2023-07-08T12:38:12.587833+0000","last_active":"2023-07-08T12:38:13.758115+0000","last_peered":"2023-07-08T12:38:13.758115+0000","last_clean":"2023-07-08T12:38:13.758115+0000","last_became_active":"2023-07-08T12:38:12.587465+0000","last_became_peered":"2023-07-08T12:38:12.587465+0000","last_unstale":"2023-07-08T12:38:13.758115+0000","last_undegraded":"2023-07-08T12:38:13.758115+0000","last_fullsized":"2023-07-08T12:38:13.758115+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:32.048081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759111+0000","last_change":"2023-07-08T12:38:02.385388+0000","last_active":"2023-07-08T12:38:13.759111+0000","last_peered":"2023-07-08T12:38:13.759111+0000","last_clean":"2023-07-08T12:38:13.759111+0000","last_became_active":"2023-07-08T12:38:02.385110+0000","last_became_peered":"2023-07-08T12:38:02.385110+0000","last_unstale":"2023-07-08T12:38:13.759111+0000","last_undegraded":"2023-07-08T12:38:13.759111+0000","last_fullsized":"2023-07-08T12:38:13.759111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:52:02.887135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759215+0000","last_change":"2023-07-08T12:38:07.544210+0000","last_active":"2023-07-08T12:38:13.759215+0000","last_peered":"2023-07-08T12:38:13.759215+0000","last_clean":"2023-07-08T12:38:13.759215+0000","last_became_active":"2023-07-08T12:38:07.543870+0000","last_became_peered":"2023-07-08T12:38:07.543870+0000","last_unstale":"2023-07-08T12:38:13.759215+0000","last_undegraded":"2023-07-08T12:38:13.759215+0000","last_fullsized":"2023-07-08T12:38:13.759215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:27.000382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606243+0000","last_change":"2023-07-08T12:38:08.548933+0000","last_active":"2023-07-08T12:38:13.606243+0000","last_peered":"2023-07-08T12:38:13.606243+0000","last_clean":"2023-07-08T12:38:13.606243+0000","last_became_active":"2023-07-08T12:38:08.548655+0000","last_became_peered":"2023-07-08T12:38:08.548655+0000","last_unstale":"2023-07-08T12:38:13.606243+0000","last_undegraded":"2023-07-08T12:38:13.606243+0000","last_fullsized":"2023-07-08T12:38:13.606243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:33:14.696652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759816+0000","last_change":"2023-07-08T12:38:12.597637+0000","last_active":"2023-07-08T12:38:13.759816+0000","last_peered":"2023-07-08T12:38:13.759816+0000","last_clean":"2023-07-08T12:38:13.759816+0000","last_became_active":"2023-07-08T12:38:12.597184+0000","last_became_peered":"2023-07-08T12:38:12.597184+0000","last_unstale":"2023-07-08T12:38:13.759816+0000","last_undegraded":"2023-07-08T12:38:13.759816+0000","last_fullsized":"2023-07-08T12:38:13.759816+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:38:44.677613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759341+0000","last_change":"2023-07-08T12:38:07.546269+0000","last_active":"2023-07-08T12:38:13.759341+0000","last_peered":"2023-07-08T12:38:13.759341+0000","last_clean":"2023-07-08T12:38:13.759341+0000","last_became_active":"2023-07-08T12:38:07.545942+0000","last_became_peered":"2023-07-08T12:38:07.545942+0000","last_unstale":"2023-07-08T12:38:13.759341+0000","last_undegraded":"2023-07-08T12:38:13.759341+0000","last_fullsized":"2023-07-08T12:38:13.759341+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:20:55.151640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604575+0000","last_change":"2023-07-08T12:38:08.566129+0000","last_active":"2023-07-08T12:38:13.604575+0000","last_peered":"2023-07-08T12:38:13.604575+0000","last_clean":"2023-07-08T12:38:13.604575+0000","last_became_active":"2023-07-08T12:38:08.565780+0000","last_became_peered":"2023-07-08T12:38:08.565780+0000","last_unstale":"2023-07-08T12:38:13.604575+0000","last_undegraded":"2023-07-08T12:38:13.604575+0000","last_fullsized":"2023-07-08T12:38:13.604575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:52:43.348476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758357+0000","last_change":"2023-07-08T12:38:12.582590+0000","last_active":"2023-07-08T12:38:13.758357+0000","last_peered":"2023-07-08T12:38:13.758357+0000","last_clean":"2023-07-08T12:38:13.758357+0000","last_became_active":"2023-07-08T12:38:12.582179+0000","last_became_peered":"2023-07-08T12:38:12.582179+0000","last_unstale":"2023-07-08T12:38:13.758357+0000","last_undegraded":"2023-07-08T12:38:13.758357+0000","last_fullsized":"2023-07-08T12:38:13.758357+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:01:39.801225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604721+0000","last_change":"2023-07-08T12:38:07.544462+0000","last_active":"2023-07-08T12:38:13.604721+0000","last_peered":"2023-07-08T12:38:13.604721+0000","last_clean":"2023-07-08T12:38:13.604721+0000","last_became_active":"2023-07-08T12:38:07.544171+0000","last_became_peered":"2023-07-08T12:38:07.544171+0000","last_unstale":"2023-07-08T12:38:13.604721+0000","last_undegraded":"2023-07-08T12:38:13.604721+0000","last_fullsized":"2023-07-08T12:38:13.604721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:14.285892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605091+0000","last_change":"2023-07-08T12:38:08.557546+0000","last_active":"2023-07-08T12:38:13.605091+0000","last_peered":"2023-07-08T12:38:13.605091+0000","last_clean":"2023-07-08T12:38:13.605091+0000","last_became_active":"2023-07-08T12:38:08.557262+0000","last_became_peered":"2023-07-08T12:38:08.557262+0000","last_unstale":"2023-07-08T12:38:13.605091+0000","last_undegraded":"2023-07-08T12:38:13.605091+0000","last_fullsized":"2023-07-08T12:38:13.605091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:11:12.440266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.605361+0000","last_change":"2023-07-08T12:38:12.597319+0000","last_active":"2023-07-08T12:38:13.605361+0000","last_peered":"2023-07-08T12:38:13.605361+0000","last_clean":"2023-07-08T12:38:13.605361+0000","last_became_active":"2023-07-08T12:38:12.596928+0000","last_became_peered":"2023-07-08T12:38:12.596928+0000","last_unstale":"2023-07-08T12:38:13.605361+0000","last_undegraded":"2023-07-08T12:38:13.605361+0000","last_fullsized":"2023-07-08T12:38:13.605361+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:39.375709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604658+0000","last_change":"2023-07-08T12:38:07.540676+0000","last_active":"2023-07-08T12:38:13.604658+0000","last_peered":"2023-07-08T12:38:13.604658+0000","last_clean":"2023-07-08T12:38:13.604658+0000","last_became_active":"2023-07-08T12:38:07.540389+0000","last_became_peered":"2023-07-08T12:38:07.540389+0000","last_unstale":"2023-07-08T12:38:13.604658+0000","last_undegraded":"2023-07-08T12:38:13.604658+0000","last_fullsized":"2023-07-08T12:38:13.604658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:05:27.000382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606089+0000","last_change":"2023-07-08T12:38:08.546186+0000","last_active":"2023-07-08T12:38:13.606089+0000","last_peered":"2023-07-08T12:38:13.606089+0000","last_clean":"2023-07-08T12:38:13.606089+0000","last_became_active":"2023-07-08T12:38:08.545902+0000","last_became_peered":"2023-07-08T12:38:08.545902+0000","last_unstale":"2023-07-08T12:38:13.606089+0000","last_undegraded":"2023-07-08T12:38:13.606089+0000","last_fullsized":"2023-07-08T12:38:13.606089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:37:20.499952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.604484+0000","last_change":"2023-07-08T12:38:12.581579+0000","last_active":"2023-07-08T12:38:13.604484+0000","last_peered":"2023-07-08T12:38:13.604484+0000","last_clean":"2023-07-08T12:38:13.604484+0000","last_became_active":"2023-07-08T12:38:12.581118+0000","last_became_peered":"2023-07-08T12:38:12.581118+0000","last_unstale":"2023-07-08T12:38:13.604484+0000","last_undegraded":"2023-07-08T12:38:13.604484+0000","last_fullsized":"2023-07-08T12:38:13.604484+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:48:23.748317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758339+0000","last_change":"2023-07-08T12:38:07.541066+0000","last_active":"2023-07-08T12:38:13.758339+0000","last_peered":"2023-07-08T12:38:13.758339+0000","last_clean":"2023-07-08T12:38:13.758339+0000","last_became_active":"2023-07-08T12:38:07.540873+0000","last_became_peered":"2023-07-08T12:38:07.540873+0000","last_unstale":"2023-07-08T12:38:13.758339+0000","last_undegraded":"2023-07-08T12:38:13.758339+0000","last_fullsized":"2023-07-08T12:38:13.758339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:54:34.328010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758518+0000","last_change":"2023-07-08T12:38:08.553600+0000","last_active":"2023-07-08T12:38:13.758518+0000","last_peered":"2023-07-08T12:38:13.758518+0000","last_clean":"2023-07-08T12:38:13.758518+0000","last_became_active":"2023-07-08T12:38:08.552418+0000","last_became_peered":"2023-07-08T12:38:08.552418+0000","last_unstale":"2023-07-08T12:38:13.758518+0000","last_undegraded":"2023-07-08T12:38:13.758518+0000","last_fullsized":"2023-07-08T12:38:13.758518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:36:03.651816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758948+0000","last_change":"2023-07-08T12:38:12.590123+0000","last_active":"2023-07-08T12:38:13.758948+0000","last_peered":"2023-07-08T12:38:13.758948+0000","last_clean":"2023-07-08T12:38:13.758948+0000","last_became_active":"2023-07-08T12:38:12.589586+0000","last_became_peered":"2023-07-08T12:38:12.589586+0000","last_unstale":"2023-07-08T12:38:13.758948+0000","last_undegraded":"2023-07-08T12:38:13.758948+0000","last_fullsized":"2023-07-08T12:38:13.758948+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:11.558469+0000","last_clean_scrub_stamp":"2023-07-08T12:38:11.558469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:03:21.918481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759739+0000","last_change":"2023-07-08T12:38:07.541648+0000","last_active":"2023-07-08T12:38:13.759739+0000","last_peered":"2023-07-08T12:38:13.759739+0000","last_clean":"2023-07-08T12:38:13.759739+0000","last_became_active":"2023-07-08T12:38:07.541284+0000","last_became_peered":"2023-07-08T12:38:07.541284+0000","last_unstale":"2023-07-08T12:38:13.759739+0000","last_undegraded":"2023-07-08T12:38:13.759739+0000","last_fullsized":"2023-07-08T12:38:13.759739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:06.510770+0000","last_clean_scrub_stamp":"2023-07-08T12:38:06.510770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:09:36.285500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606128+0000","last_change":"2023-07-08T12:38:08.552907+0000","last_active":"2023-07-08T12:38:13.606128+0000","last_peered":"2023-07-08T12:38:13.606128+0000","last_clean":"2023-07-08T12:38:13.606128+0000","last_became_active":"2023-07-08T12:38:08.552595+0000","last_became_peered":"2023-07-08T12:38:08.552595+0000","last_unstale":"2023-07-08T12:38:13.606128+0000","last_undegraded":"2023-07-08T12:38:13.606128+0000","last_fullsized":"2023-07-08T12:38:13.606128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:02:09.276408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759378+0000","last_change":"2023-07-08T12:38:08.554379+0000","last_active":"2023-07-08T12:38:13.759378+0000","last_peered":"2023-07-08T12:38:13.759378+0000","last_clean":"2023-07-08T12:38:13.759378+0000","last_became_active":"2023-07-08T12:38:08.554092+0000","last_became_peered":"2023-07-08T12:38:08.554092+0000","last_unstale":"2023-07-08T12:38:13.759378+0000","last_undegraded":"2023-07-08T12:38:13.759378+0000","last_fullsized":"2023-07-08T12:38:13.759378+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:36:56.254060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758785+0000","last_change":"2023-07-08T12:38:08.557447+0000","last_active":"2023-07-08T12:38:13.758785+0000","last_peered":"2023-07-08T12:38:13.758785+0000","last_clean":"2023-07-08T12:38:13.758785+0000","last_became_active":"2023-07-08T12:38:08.557112+0000","last_became_peered":"2023-07-08T12:38:08.557112+0000","last_unstale":"2023-07-08T12:38:13.758785+0000","last_undegraded":"2023-07-08T12:38:13.758785+0000","last_fullsized":"2023-07-08T12:38:13.758785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:31:53.990746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.606414+0000","last_change":"2023-07-08T12:38:08.550996+0000","last_active":"2023-07-08T12:38:13.606414+0000","last_peered":"2023-07-08T12:38:13.606414+0000","last_clean":"2023-07-08T12:38:13.606414+0000","last_became_active":"2023-07-08T12:38:08.550597+0000","last_became_peered":"2023-07-08T12:38:08.550597+0000","last_unstale":"2023-07-08T12:38:13.606414+0000","last_undegraded":"2023-07-08T12:38:13.606414+0000","last_fullsized":"2023-07-08T12:38:13.606414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:44:57.917258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.758464+0000","last_change":"2023-07-08T12:38:08.552118+0000","last_active":"2023-07-08T12:38:13.758464+0000","last_peered":"2023-07-08T12:38:13.758464+0000","last_clean":"2023-07-08T12:38:13.758464+0000","last_became_active":"2023-07-08T12:38:08.551258+0000","last_became_peered":"2023-07-08T12:38:08.551258+0000","last_unstale":"2023-07-08T12:38:13.758464+0000","last_undegraded":"2023-07-08T12:38:13.758464+0000","last_fullsized":"2023-07-08T12:38:13.758464+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:29:22.667819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-07-08T12:38:13.759628+0000","last_change":"2023-07-08T12:38:08.563050+0000","last_active":"2023-07-08T12:38:13.759628+0000","last_peered":"2023-07-08T12:38:13.759628+0000","last_clean":"2023-07-08T12:38:13.759628+0000","last_became_active":"2023-07-08T12:38:08.562779+0000","last_became_peered":"2023-07-08T12:38:08.562779+0000","last_unstale":"2023-07-08T12:38:13.759628+0000","last_undegraded":"2023-07-08T12:38:13.759628+0000","last_fullsized":"2023-07-08T12:38:13.759628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:07.515370+0000","last_clean_scrub_stamp":"2023-07-08T12:38:07.515370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:05:15.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4592,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":718950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":901120,"data_stored":715522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":323584,"data_stored":128422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":311296,"data_stored":125224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-08T12:38:38.681 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-08T12:38:38.682 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-08T12:38:38.682 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-07-08T12:38:38.682 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-08T12:38:38.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.834+0000 7f4c5c964700 1 Processor -- start 2023-07-08T12:38:38.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.834+0000 7f4c5c964700 1 -- start start 2023-07-08T12:38:38.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5c964700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4c5815a480 0x7f4c580cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5c964700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c580b2410 0x7f4c580b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5c964700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5815a850 con 0x7f4c5815def0 2023-07-08T12:38:38.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5c964700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c580b2da0 con 0x7f4c5815a480 2023-07-08T12:38:38.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5c964700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c580cde90 con 0x7f4c580b2410 2023-07-08T12:38:38.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5659c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4c5815a480 0x7f4c580cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5659c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4c5815a480 0x7f4c580cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50024/0 (socket says 172.21.15.12:50024) 2023-07-08T12:38:38.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5659c700 1 -- 172.21.15.12:0/2746359415 learned_addr learned my addr 172.21.15.12:0/2746359415 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:38.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c55d9b700 1 --2- 172.21.15.12:0/2746359415 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c580b2410 0x7f4c580b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5659c700 1 -- 172.21.15.12:0/2746359415 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c580b2410 msgr2=0x7f4c580b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:38.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5659c700 1 --2- 172.21.15.12:0/2746359415 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c580b2410 0x7f4c580b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5659c700 1 -- 172.21.15.12:0/2746359415 >> v1:172.21.15.160:6789/0 conn(0x7f4c5815def0 legacy=0x7f4c5815e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5659c700 1 -- 172.21.15.12:0/2746359415 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c580cd8d0 con 0x7f4c5815a480 2023-07-08T12:38:38.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5659c700 1 --2- 172.21.15.12:0/2746359415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c5815a480 0x7f4c580cafe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4c480098f0 tx=0x7f4c48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62464f42630c224 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:38.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.835+0000 7f4c5559a700 1 -- 172.21.15.12:0/2746359415 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4800b8a0 con 0x7f4c5815a480 2023-07-08T12:38:38.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.836+0000 7f4c5559a700 1 -- 172.21.15.12:0/2746359415 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4800ba00 con 0x7f4c5815a480 2023-07-08T12:38:38.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.836+0000 7f4c5559a700 1 -- 172.21.15.12:0/2746359415 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4800bd90 con 0x7f4c5815a480 2023-07-08T12:38:38.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.836+0000 7f4c5c964700 1 -- 172.21.15.12:0/2746359415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c5815a480 msgr2=0x7f4c580cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.836+0000 7f4c5c964700 1 --2- 172.21.15.12:0/2746359415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c5815a480 0x7f4c580cafe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4c480098f0 tx=0x7f4c48009c00 comp rx=0 tx=0).stop 2023-07-08T12:38:38.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.836+0000 7f4c5c964700 1 -- 172.21.15.12:0/2746359415 shutdown_connections 2023-07-08T12:38:38.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.836+0000 7f4c5c964700 1 --2- 172.21.15.12:0/2746359415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c5815a480 0x7f4c580cafe0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.836+0000 7f4c5c964700 1 --2- 172.21.15.12:0/2746359415 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c580b2410 0x7f4c580b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.836+0000 7f4c5c964700 1 -- 172.21.15.12:0/2746359415 >> 172.21.15.12:0/2746359415 conn(0x7f4c5806ab60 msgr2=0x7f4c5806e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:38.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.836+0000 7f4c5c964700 1 -- 172.21.15.12:0/2746359415 shutdown_connections 2023-07-08T12:38:38.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.836+0000 7f4c5c964700 1 -- 172.21.15.12:0/2746359415 wait complete. 2023-07-08T12:38:38.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.838+0000 7f4c5c964700 1 Processor -- start 2023-07-08T12:38:38.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.838+0000 7f4c5c964700 1 -- start start 2023-07-08T12:38:38.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.838+0000 7f4c5c964700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c580b2410 0x7f4c581efb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.838+0000 7f4c5c964700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c5815a480 0x7f4c58157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.838+0000 7f4c5c964700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c5815def0 0x7f4c581f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.838+0000 7f4c5c964700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c581f4650 con 0x7f4c580b2410 2023-07-08T12:38:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.838+0000 7f4c5c964700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c580b3d80 con 0x7f4c5815def0 2023-07-08T12:38:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.838+0000 7f4c5c964700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c58146630 con 0x7f4c5815a480 2023-07-08T12:38:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.838+0000 7f4c5659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c580b2410 0x7f4c581efb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.838+0000 7f4c5659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c580b2410 0x7f4c581efb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50030/0 (socket says 172.21.15.12:50030) 2023-07-08T12:38:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.838+0000 7f4c5659c700 1 -- 172.21.15.12:0/741312895 learned_addr learned my addr 172.21.15.12:0/741312895 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.839+0000 7f4c56d9d700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c5815def0 0x7f4c581f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.839+0000 7f4c55d9b700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c5815a480 0x7f4c58157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.839+0000 7f4c5659c700 1 -- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c5815a480 msgr2=0x7f4c58157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.839+0000 7f4c5659c700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c5815a480 0x7f4c58157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.839+0000 7f4c5659c700 1 -- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c5815def0 msgr2=0x7f4c581f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:38.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.839+0000 7f4c5659c700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c5815def0 0x7f4c581f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:38.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.839+0000 7f4c5659c700 1 -- 172.21.15.12:0/741312895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c48009550 con 0x7f4c580b2410 2023-07-08T12:38:38.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.839+0000 7f4c5659c700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c580b2410 0x7f4c581efb10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4c48014040 tx=0x7f4c48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:38.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.839+0000 7f4c46ffd700 1 -- 172.21.15.12:0/741312895 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c480036a0 con 0x7f4c580b2410 2023-07-08T12:38:38.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.839+0000 7f4c46ffd700 1 -- 172.21.15.12:0/741312895 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4801e070 con 0x7f4c580b2410 2023-07-08T12:38:38.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.839+0000 7f4c46ffd700 1 -- 172.21.15.12:0/741312895 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48003a40 con 0x7f4c580b2410 2023-07-08T12:38:38.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.839+0000 7f4c5c964700 1 -- 172.21.15.12:0/741312895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c581f0050 con 0x7f4c580b2410 2023-07-08T12:38:38.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.839+0000 7f4c5c964700 1 -- 172.21.15.12:0/741312895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c58156ca0 con 0x7f4c580b2410 2023-07-08T12:38:38.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.840+0000 7f4c5c964700 1 -- 172.21.15.12:0/741312895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5804ea60 con 0x7f4c580b2410 2023-07-08T12:38:38.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.840+0000 7f4c46ffd700 1 -- 172.21.15.12:0/741312895 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c48003ba0 con 0x7f4c580b2410 2023-07-08T12:38:38.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.841+0000 7f4c46ffd700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4c3806ace0 0x7f4c3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:38.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.841+0000 7f4c46ffd700 1 -- 172.21.15.12:0/741312895 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4c48020070 con 0x7f4c580b2410 2023-07-08T12:38:38.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.841+0000 7f4c55d9b700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4c3806ace0 0x7f4c3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:38.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.842+0000 7f4c55d9b700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4c3806ace0 0x7f4c3806d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4c581e48c0 tx=0x7f4c4c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:38.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:38.845+0000 7f4c46ffd700 1 -- 172.21.15.12:0/741312895 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4c48055a40 con 0x7f4c580b2410 2023-07-08T12:38:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.088+0000 7f4c5c964700 1 -- 172.21.15.12:0/741312895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4c580729d0 con 0x7f4c580b2410 2023-07-08T12:38:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.088+0000 7f4c46ffd700 1 -- 172.21.15.12:0/741312895 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4c48059060 con 0x7f4c580b2410 2023-07-08T12:38:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:39.090 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-08T12:38:39.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.090+0000 7f4c5c964700 1 -- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4c3806ace0 msgr2=0x7f4c3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.090+0000 7f4c5c964700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4c3806ace0 0x7f4c3806d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4c581e48c0 tx=0x7f4c4c008040 comp rx=0 tx=0).stop 2023-07-08T12:38:39.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.090+0000 7f4c5c964700 1 -- 172.21.15.12:0/741312895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c580b2410 msgr2=0x7f4c581efb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.090+0000 7f4c5c964700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c580b2410 0x7f4c581efb10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4c48014040 tx=0x7f4c48004750 comp rx=0 tx=0).stop 2023-07-08T12:38:39.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.090+0000 7f4c5c964700 1 -- 172.21.15.12:0/741312895 shutdown_connections 2023-07-08T12:38:39.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.090+0000 7f4c5c964700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4c3806ace0 0x7f4c3806d190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.090+0000 7f4c5c964700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c5815def0 0x7f4c581f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.090+0000 7f4c5c964700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c580b2410 0x7f4c581efb10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.090+0000 7f4c5c964700 1 --2- 172.21.15.12:0/741312895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c5815a480 0x7f4c58157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.090+0000 7f4c5c964700 1 -- 172.21.15.12:0/741312895 >> 172.21.15.12:0/741312895 conn(0x7f4c5806ab60 msgr2=0x7f4c580b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:39.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.090+0000 7f4c5c964700 1 -- 172.21.15.12:0/741312895 shutdown_connections 2023-07-08T12:38:39.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.090+0000 7f4c5c964700 1 -- 172.21.15.12:0/741312895 wait complete. 2023-07-08T12:38:39.104 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-07-08T12:38:39.105 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:39.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.227+0000 7f8801eb8700 1 Processor -- start 2023-07-08T12:38:39.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.227+0000 7f8801eb8700 1 -- start start 2023-07-08T12:38:39.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.228+0000 7f8801eb8700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87fc0b1c70 0x7f87fc0ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.228+0000 7f8801eb8700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f87fc15d6c0 0x7f87fc071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.228+0000 7f8801eb8700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87fc15d230 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.228+0000 7f8801eb8700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87fc0c9e60 con 0x7f87fc15d6c0 2023-07-08T12:38:39.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.228+0000 7f8801eb8700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc0ceac0 con 0x7f87fc0b1c70 2023-07-08T12:38:39.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87faffd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87fc0b1c70 0x7f87fc0ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87fbfff700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f87fc15d6c0 0x7f87fc071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87fb7fe700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f87fc0ce1b0 0x7f87fc0cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:46656/0 (socket says 172.21.15.12:46656) 2023-07-08T12:38:39.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87faffd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87fc0b1c70 0x7f87fc0ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59658/0 (socket says 172.21.15.12:59658) 2023-07-08T12:38:39.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87fb7fe700 1 -- 172.21.15.12:0/3831298932 learned_addr learned my addr 172.21.15.12:0/3831298932 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:39.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87faffd700 1 -- 172.21.15.12:0/3831298932 >> v2:172.21.15.160:3300/0 conn(0x7f87fc15d6c0 msgr2=0x7f87fc071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:39.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87faffd700 1 --2- 172.21.15.12:0/3831298932 >> v2:172.21.15.160:3300/0 conn(0x7f87fc15d6c0 0x7f87fc071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87faffd700 1 -- 172.21.15.12:0/3831298932 >> v1:172.21.15.160:6789/0 conn(0x7f87fc0ce1b0 legacy=0x7f87fc0cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87faffd700 1 -- 172.21.15.12:0/3831298932 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc0cdbc0 con 0x7f87fc0b1c70 2023-07-08T12:38:39.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87fbfff700 1 --2- 172.21.15.12:0/3831298932 >> v2:172.21.15.160:3300/0 conn(0x7f87fc15d6c0 0x7f87fc071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:38:39.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87faffd700 1 --2- 172.21.15.12:0/3831298932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87fc0b1c70 0x7f87fc0ce580 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009840 tx=0x7f87f0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=253c8c1b3f32fa4c server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:39.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87fa7fc700 1 -- 172.21.15.12:0/3831298932 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0011830 con 0x7f87fc0b1c70 2023-07-08T12:38:39.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87fa7fc700 1 -- 172.21.15.12:0/3831298932 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f0011990 con 0x7f87fc0b1c70 2023-07-08T12:38:39.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.229+0000 7f87fa7fc700 1 -- 172.21.15.12:0/3831298932 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0011d20 con 0x7f87fc0b1c70 2023-07-08T12:38:39.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.230+0000 7f8801eb8700 1 -- 172.21.15.12:0/3831298932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87fc0b1c70 msgr2=0x7f87fc0ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.230+0000 7f8801eb8700 1 --2- 172.21.15.12:0/3831298932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87fc0b1c70 0x7f87fc0ce580 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009840 tx=0x7f87f0009b50 comp rx=0 tx=0).stop 2023-07-08T12:38:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.230+0000 7f8801eb8700 1 -- 172.21.15.12:0/3831298932 shutdown_connections 2023-07-08T12:38:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.230+0000 7f8801eb8700 1 --2- 172.21.15.12:0/3831298932 >> v2:172.21.15.160:3300/0 conn(0x7f87fc15d6c0 0x7f87fc071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.230+0000 7f8801eb8700 1 --2- 172.21.15.12:0/3831298932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87fc0b1c70 0x7f87fc0ce580 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.230+0000 7f8801eb8700 1 -- 172.21.15.12:0/3831298932 >> 172.21.15.12:0/3831298932 conn(0x7f87fc0cc290 msgr2=0x7f87fc0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:39.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.230+0000 7f8801eb8700 1 -- 172.21.15.12:0/3831298932 shutdown_connections 2023-07-08T12:38:39.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.230+0000 7f8801eb8700 1 -- 172.21.15.12:0/3831298932 wait complete. 2023-07-08T12:38:39.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.231+0000 7f8801eb8700 1 Processor -- start 2023-07-08T12:38:39.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.231+0000 7f8801eb8700 1 -- start start 2023-07-08T12:38:39.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.232+0000 7f8801eb8700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87fc0b1c70 0x7f87fc1ee050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.232+0000 7f8801eb8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0ce1b0 0x7f87fc157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.232+0000 7f8801eb8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87fc15d6c0 0x7f87fc1f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.232+0000 7f8801eb8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc1f1b60 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.232+0000 7f8801eb8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc1ed640 con 0x7f87fc15d6c0 2023-07-08T12:38:39.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.232+0000 7f8801eb8700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc1ee590 con 0x7f87fc0b1c70 2023-07-08T12:38:39.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.232+0000 7f87faffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0ce1b0 0x7f87fc157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.232+0000 7f87faffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0ce1b0 0x7f87fc157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50046/0 (socket says 172.21.15.12:50046) 2023-07-08T12:38:39.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.232+0000 7f87faffd700 1 -- 172.21.15.12:0/1299800699 learned_addr learned my addr 172.21.15.12:0/1299800699 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:39.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.232+0000 7f87fb7fe700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87fc0b1c70 0x7f87fc1ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.232+0000 7f87fbfff700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87fc15d6c0 0x7f87fc1f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.233+0000 7f87faffd700 1 -- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87fc0b1c70 msgr2=0x7f87fc1ee050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.233+0000 7f87faffd700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87fc0b1c70 0x7f87fc1ee050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.233+0000 7f87faffd700 1 -- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87fc15d6c0 msgr2=0x7f87fc1f0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.233+0000 7f87faffd700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87fc15d6c0 0x7f87fc1f0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.233+0000 7f87faffd700 1 -- 172.21.15.12:0/1299800699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f4009a70 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.233+0000 7f87fbfff700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87fc15d6c0 0x7f87fc1f0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:39.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.233+0000 7f87faffd700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0ce1b0 0x7f87fc157fc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009fd0 tx=0x7f87f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:39.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.233+0000 7f87e3fff700 1 -- 172.21.15.12:0/1299800699 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f00036a0 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.233+0000 7f87e3fff700 1 -- 172.21.15.12:0/1299800699 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f001e070 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.233+0000 7f8801eb8700 1 -- 172.21.15.12:0/1299800699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f00094a0 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.233+0000 7f87e3fff700 1 -- 172.21.15.12:0/1299800699 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0003a40 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.233+0000 7f8801eb8700 1 -- 172.21.15.12:0/1299800699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc1f15a0 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.234+0000 7f8801eb8700 1 -- 172.21.15.12:0/1299800699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87fc066e80 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.234+0000 7f87e3fff700 1 -- 172.21.15.12:0/1299800699 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f87f0003ba0 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.235+0000 7f87e3fff700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f87e806ace0 0x7f87e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.235+0000 7f87e3fff700 1 -- 172.21.15.12:0/1299800699 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f87f008a6e0 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.235+0000 7f87fb7fe700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f87e806ace0 0x7f87e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.236+0000 7f87fb7fe700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f87e806ace0 0x7f87e806d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f87ec005910 tx=0x7f87ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:39.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.239+0000 7f87e3fff700 1 -- 172.21.15.12:0/1299800699 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f87f0055530 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.437+0000 7f8801eb8700 1 -- 172.21.15.12:0/1299800699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87fc04ea60 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.438+0000 7f87e3fff700 1 -- 172.21.15.12:0/1299800699 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f87f0058b50 con 0x7f87fc0ce1b0 2023-07-08T12:38:39.441 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:39.441 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:12.993803+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.440+0000 7f8801eb8700 1 -- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f87e806ace0 msgr2=0x7f87e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.440+0000 7f8801eb8700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f87e806ace0 0x7f87e806d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f87ec005910 tx=0x7f87ec009040 comp rx=0 tx=0).stop 2023-07-08T12:38:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.441+0000 7f8801eb8700 1 -- 172.21.15.12:0/1299800699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0ce1b0 msgr2=0x7f87fc157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.441+0000 7f8801eb8700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0ce1b0 0x7f87fc157fc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009fd0 tx=0x7f87f0004750 comp rx=0 tx=0).stop 2023-07-08T12:38:39.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.441+0000 7f8801eb8700 1 -- 172.21.15.12:0/1299800699 shutdown_connections 2023-07-08T12:38:39.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.441+0000 7f8801eb8700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f87e806ace0 0x7f87e806d190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.441+0000 7f8801eb8700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87fc15d6c0 0x7f87fc1f0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.441+0000 7f8801eb8700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87fc0b1c70 0x7f87fc1ee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.441+0000 7f8801eb8700 1 --2- 172.21.15.12:0/1299800699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0ce1b0 0x7f87fc157fc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.441+0000 7f8801eb8700 1 -- 172.21.15.12:0/1299800699 >> 172.21.15.12:0/1299800699 conn(0x7f87fc0cc290 msgr2=0x7f87fc0cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:39.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.441+0000 7f8801eb8700 1 -- 172.21.15.12:0/1299800699 shutdown_connections 2023-07-08T12:38:39.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.441+0000 7f8801eb8700 1 -- 172.21.15.12:0/1299800699 wait complete. 2023-07-08T12:38:39.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-08T12:38:05.660194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-08T12:38:06.920177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-08T12:38:10.592749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-08T12:38:39.456 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:38:39.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.584+0000 7f39d8f8a700 1 Processor -- start 2023-07-08T12:38:39.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.584+0000 7f39d8f8a700 1 -- start start 2023-07-08T12:38:39.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.585+0000 7f39d8f8a700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f39d40ce560 0x7f39d40ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.585+0000 7f39d8f8a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39d4161ce0 0x7f39d41640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.585+0000 7f39d8f8a700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d4158130 con 0x7f39d415a6d0 2023-07-08T12:38:39.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.585+0000 7f39d8f8a700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d415b730 con 0x7f39d40ce560 2023-07-08T12:38:39.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.585+0000 7f39d8f8a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d415b230 con 0x7f39d4161ce0 2023-07-08T12:38:39.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d2d9d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39d4161ce0 0x7f39d41640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d259c700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f39d40ce560 0x7f39d40ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d259c700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f39d40ce560 0x7f39d40ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:48796/0 (socket says 172.21.15.12:48796) 2023-07-08T12:38:39.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d1d9b700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f39d415a6d0 0x7f39d415ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:46670/0 (socket says 172.21.15.12:46670) 2023-07-08T12:38:39.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d259c700 1 -- 172.21.15.12:0/2863639283 learned_addr learned my addr 172.21.15.12:0/2863639283 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:39.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d2d9d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39d4161ce0 0x7f39d41640c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:59674/0 (socket says 172.21.15.12:59674) 2023-07-08T12:38:39.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d259c700 1 -- 172.21.15.12:0/2863639283 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39d4161ce0 msgr2=0x7f39d41640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:39.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d259c700 1 --2- 172.21.15.12:0/2863639283 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39d4161ce0 0x7f39d41640c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d259c700 1 -- 172.21.15.12:0/2863639283 >> v1:172.21.15.160:6789/0 conn(0x7f39d415a6d0 legacy=0x7f39d415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d259c700 1 -- 172.21.15.12:0/2863639283 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d40cee70 con 0x7f39d40ce560 2023-07-08T12:38:39.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d259c700 1 --2- 172.21.15.12:0/2863639283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d40ce560 0x7f39d40ce930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f39c80098f0 tx=0x7f39c8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db96f5bfb967bb3c server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:39.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d159a700 1 -- 172.21.15.12:0/2863639283 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8011830 con 0x7f39d40ce560 2023-07-08T12:38:39.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d159a700 1 -- 172.21.15.12:0/2863639283 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c8011990 con 0x7f39d40ce560 2023-07-08T12:38:39.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.586+0000 7f39d159a700 1 -- 172.21.15.12:0/2863639283 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8011d20 con 0x7f39d40ce560 2023-07-08T12:38:39.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.587+0000 7f39d8f8a700 1 -- 172.21.15.12:0/2863639283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d40ce560 msgr2=0x7f39d40ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.587+0000 7f39d8f8a700 1 --2- 172.21.15.12:0/2863639283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d40ce560 0x7f39d40ce930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f39c80098f0 tx=0x7f39c8009c00 comp rx=0 tx=0).stop 2023-07-08T12:38:39.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.587+0000 7f39d8f8a700 1 -- 172.21.15.12:0/2863639283 shutdown_connections 2023-07-08T12:38:39.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.587+0000 7f39d8f8a700 1 --2- 172.21.15.12:0/2863639283 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39d4161ce0 0x7f39d41640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.587+0000 7f39d8f8a700 1 --2- 172.21.15.12:0/2863639283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d40ce560 0x7f39d40ce930 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.587+0000 7f39d8f8a700 1 -- 172.21.15.12:0/2863639283 >> 172.21.15.12:0/2863639283 conn(0x7f39d4099c60 msgr2=0x7f39d4089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:39.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.587+0000 7f39d8f8a700 1 -- 172.21.15.12:0/2863639283 shutdown_connections 2023-07-08T12:38:39.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.587+0000 7f39d8f8a700 1 -- 172.21.15.12:0/2863639283 wait complete. 2023-07-08T12:38:39.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.588+0000 7f39d8f8a700 1 Processor -- start 2023-07-08T12:38:39.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.588+0000 7f39d8f8a700 1 -- start start 2023-07-08T12:38:39.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.589+0000 7f39d8f8a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39d40ce560 0x7f39d4153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.589+0000 7f39d8f8a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d415a6d0 0x7f39d40c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.589+0000 7f39d8f8a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d4161ce0 0x7f39d40c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.589+0000 7f39d8f8a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d4146250 con 0x7f39d415a6d0 2023-07-08T12:38:39.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.589+0000 7f39d8f8a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40b14e0 con 0x7f39d4161ce0 2023-07-08T12:38:39.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.589+0000 7f39d8f8a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40b46c0 con 0x7f39d40ce560 2023-07-08T12:38:39.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.589+0000 7f39d1d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d415a6d0 0x7f39d40c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.589+0000 7f39d1d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d415a6d0 0x7f39d40c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50060/0 (socket says 172.21.15.12:50060) 2023-07-08T12:38:39.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.589+0000 7f39d1d9b700 1 -- 172.21.15.12:0/3372318765 learned_addr learned my addr 172.21.15.12:0/3372318765 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:39.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.589+0000 7f39d259c700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39d40ce560 0x7f39d4153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.589+0000 7f39d2d9d700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d4161ce0 0x7f39d40c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.590+0000 7f39d1d9b700 1 -- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39d40ce560 msgr2=0x7f39d4153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.590+0000 7f39d1d9b700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39d40ce560 0x7f39d4153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.590+0000 7f39d1d9b700 1 -- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d4161ce0 msgr2=0x7f39d40c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.590+0000 7f39d1d9b700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d4161ce0 0x7f39d40c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.590+0000 7f39d1d9b700 1 -- 172.21.15.12:0/3372318765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c8009550 con 0x7f39d415a6d0 2023-07-08T12:38:39.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.590+0000 7f39d2d9d700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d4161ce0 0x7f39d40c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:38:39.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.590+0000 7f39d1d9b700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d415a6d0 0x7f39d40c64d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f39c400c500 tx=0x7f39c400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:39.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.590+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3372318765 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4005cb0 con 0x7f39d415a6d0 2023-07-08T12:38:39.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.590+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3372318765 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c4005e10 con 0x7f39d415a6d0 2023-07-08T12:38:39.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.590+0000 7f39d8f8a700 1 -- 172.21.15.12:0/3372318765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d40b1780 con 0x7f39d415a6d0 2023-07-08T12:38:39.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.590+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3372318765 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c40072d0 con 0x7f39d415a6d0 2023-07-08T12:38:39.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.590+0000 7f39d8f8a700 1 -- 172.21.15.12:0/3372318765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d40b1d40 con 0x7f39d415a6d0 2023-07-08T12:38:39.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.591+0000 7f39d8f8a700 1 -- 172.21.15.12:0/3372318765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d404ea60 con 0x7f39d415a6d0 2023-07-08T12:38:39.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.591+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3372318765 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f39c4007430 con 0x7f39d415a6d0 2023-07-08T12:38:39.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.592+0000 7f39c2ffd700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f39b406ace0 0x7f39b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.592+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3372318765 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f39c4088930 con 0x7f39d415a6d0 2023-07-08T12:38:39.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.592+0000 7f39d259c700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f39b406ace0 0x7f39b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.593+0000 7f39d259c700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f39b406ace0 0x7f39b406d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f39c8014040 tx=0x7f39c8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:39.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.596+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3372318765 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f39c40537a0 con 0x7f39d415a6d0 2023-07-08T12:38:39.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.793+0000 7f39d8f8a700 1 -- 172.21.15.12:0/3372318765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39d415acc0 con 0x7f39d415a6d0 2023-07-08T12:38:39.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.794+0000 7f39c2ffd700 1 -- 172.21.15.12:0/3372318765 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39c4017020 con 0x7f39d415a6d0 2023-07-08T12:38:39.797 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:38:39.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.796+0000 7f39d8f8a700 1 -- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f39b406ace0 msgr2=0x7f39b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.796+0000 7f39d8f8a700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f39b406ace0 0x7f39b406d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f39c8014040 tx=0x7f39c8008030 comp rx=0 tx=0).stop 2023-07-08T12:38:39.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.796+0000 7f39d8f8a700 1 -- 172.21.15.12:0/3372318765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d415a6d0 msgr2=0x7f39d40c64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.796+0000 7f39d8f8a700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d415a6d0 0x7f39d40c64d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f39c400c500 tx=0x7f39c400c810 comp rx=0 tx=0).stop 2023-07-08T12:38:39.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.796+0000 7f39d8f8a700 1 -- 172.21.15.12:0/3372318765 shutdown_connections 2023-07-08T12:38:39.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.796+0000 7f39d8f8a700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f39b406ace0 0x7f39b406d190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.796+0000 7f39d8f8a700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d4161ce0 0x7f39d40c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.796+0000 7f39d8f8a700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39d40ce560 0x7f39d4153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.796+0000 7f39d8f8a700 1 --2- 172.21.15.12:0/3372318765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f39d415a6d0 0x7f39d40c64d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.796+0000 7f39d8f8a700 1 -- 172.21.15.12:0/3372318765 >> 172.21.15.12:0/3372318765 conn(0x7f39d4099c60 msgr2=0x7f39d40d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:39.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.796+0000 7f39d8f8a700 1 -- 172.21.15.12:0/3372318765 shutdown_connections 2023-07-08T12:38:39.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.796+0000 7f39d8f8a700 1 -- 172.21.15.12:0/3372318765 wait complete. 2023-07-08T12:38:39.810 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:38:39.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.935+0000 7fe1cea47700 1 Processor -- start 2023-07-08T12:38:39.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.935+0000 7fe1cea47700 1 -- start start 2023-07-08T12:38:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.936+0000 7fe1cea47700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1c80cc6d0 0x7fe1c80c9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.936+0000 7fe1cea47700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe1c8070b00 0x7fe1c8070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.936+0000 7fe1cea47700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c806a620 con 0x7fe1c8158b50 2023-07-08T12:38:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.936+0000 7fe1cea47700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c80714b0 con 0x7fe1c8070b00 2023-07-08T12:38:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.936+0000 7fe1cea47700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c815d0e0 con 0x7fe1c80cc6d0 2023-07-08T12:38:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.936+0000 7fe1ccfe4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe1c8070b00 0x7fe1c8070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.936+0000 7fe1ccfe4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe1c8070b00 0x7fe1c8070f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50070/0 (socket says 172.21.15.12:50070) 2023-07-08T12:38:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.936+0000 7fe1ccfe4700 1 -- 172.21.15.12:0/4229267762 learned_addr learned my addr 172.21.15.12:0/4229267762 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.936+0000 7fe1c77fe700 1 --2- 172.21.15.12:0/4229267762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1c80cc6d0 0x7fe1c80c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.937+0000 7fe1ccfe4700 1 -- 172.21.15.12:0/4229267762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1c80cc6d0 msgr2=0x7fe1c80c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:39.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.937+0000 7fe1ccfe4700 1 --2- 172.21.15.12:0/4229267762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1c80cc6d0 0x7fe1c80c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.937+0000 7fe1ccfe4700 1 -- 172.21.15.12:0/4229267762 >> v1:172.21.15.160:6789/0 conn(0x7fe1c8158b50 legacy=0x7fe1c80cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.937+0000 7fe1ccfe4700 1 -- 172.21.15.12:0/4229267762 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c815d3c0 con 0x7fe1c8070b00 2023-07-08T12:38:39.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.937+0000 7fe1ccfe4700 1 --2- 172.21.15.12:0/4229267762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1c8070b00 0x7fe1c8070f70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000d150 tx=0x7fe1c000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0261d10ebe7e262 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:39.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.937+0000 7fe1c6ffd700 1 -- 172.21.15.12:0/4229267762 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000b2d0 con 0x7fe1c8070b00 2023-07-08T12:38:39.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.937+0000 7fe1c6ffd700 1 -- 172.21.15.12:0/4229267762 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c0010070 con 0x7fe1c8070b00 2023-07-08T12:38:39.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.937+0000 7fe1c6ffd700 1 -- 172.21.15.12:0/4229267762 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000da40 con 0x7fe1c8070b00 2023-07-08T12:38:39.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.937+0000 7fe1cea47700 1 -- 172.21.15.12:0/4229267762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1c8070b00 msgr2=0x7fe1c8070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.937+0000 7fe1cea47700 1 --2- 172.21.15.12:0/4229267762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1c8070b00 0x7fe1c8070f70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000d150 tx=0x7fe1c000d460 comp rx=0 tx=0).stop 2023-07-08T12:38:39.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.938+0000 7fe1cea47700 1 -- 172.21.15.12:0/4229267762 shutdown_connections 2023-07-08T12:38:39.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.938+0000 7fe1cea47700 1 --2- 172.21.15.12:0/4229267762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1c8070b00 0x7fe1c8070f70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.938+0000 7fe1cea47700 1 --2- 172.21.15.12:0/4229267762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1c80cc6d0 0x7fe1c80c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.938+0000 7fe1cea47700 1 -- 172.21.15.12:0/4229267762 >> 172.21.15.12:0/4229267762 conn(0x7fe1c80cfc80 msgr2=0x7fe1c80cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:39.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.938+0000 7fe1cea47700 1 -- 172.21.15.12:0/4229267762 shutdown_connections 2023-07-08T12:38:39.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.938+0000 7fe1cea47700 1 -- 172.21.15.12:0/4229267762 wait complete. 2023-07-08T12:38:39.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.939+0000 7fe1cea47700 1 Processor -- start 2023-07-08T12:38:39.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.939+0000 7fe1cea47700 1 -- start start 2023-07-08T12:38:39.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1cea47700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1c8070b00 0x7fe1c80c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1cea47700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c80cc6d0 0x7fe1c8152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1cea47700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1c8158b50 0x7fe1c815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1cea47700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80c5e80 con 0x7fe1c8158b50 2023-07-08T12:38:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1cea47700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80c6600 con 0x7fe1c80cc6d0 2023-07-08T12:38:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1cea47700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80c4040 con 0x7fe1c8070b00 2023-07-08T12:38:39.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1ccfe4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1c8158b50 0x7fe1c815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1ccfe4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1c8158b50 0x7fe1c815c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50076/0 (socket says 172.21.15.12:50076) 2023-07-08T12:38:39.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1ccfe4700 1 -- 172.21.15.12:0/1333581491 learned_addr learned my addr 172.21.15.12:0/1333581491 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:39.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1c77fe700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c80cc6d0 0x7fe1c8152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1c7fff700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1c8070b00 0x7fe1c80c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1ccfe4700 1 -- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1c8070b00 msgr2=0x7fe1c80c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1ccfe4700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1c8070b00 0x7fe1c80c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1ccfe4700 1 -- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c80cc6d0 msgr2=0x7fe1c8152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:39.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1ccfe4700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c80cc6d0 0x7fe1c8152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:39.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1ccfe4700 1 -- 172.21.15.12:0/1333581491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c000bd40 con 0x7fe1c8158b50 2023-07-08T12:38:39.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1ccfe4700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1c8158b50 0x7fe1c815c4c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe1c806caf0 tx=0x7fe1c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:39.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1c4ff9700 1 -- 172.21.15.12:0/1333581491 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000a890 con 0x7fe1c8158b50 2023-07-08T12:38:39.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1c4ff9700 1 -- 172.21.15.12:0/1333581491 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c000a9f0 con 0x7fe1c8158b50 2023-07-08T12:38:39.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.940+0000 7fe1cea47700 1 -- 172.21.15.12:0/1333581491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c815b580 con 0x7fe1c8158b50 2023-07-08T12:38:39.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.941+0000 7fe1c4ff9700 1 -- 172.21.15.12:0/1333581491 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000ad60 con 0x7fe1c8158b50 2023-07-08T12:38:39.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.941+0000 7fe1cea47700 1 -- 172.21.15.12:0/1333581491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c80c5970 con 0x7fe1c8158b50 2023-07-08T12:38:39.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.941+0000 7fe1cea47700 1 -- 172.21.15.12:0/1333581491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c804ea60 con 0x7fe1c8158b50 2023-07-08T12:38:39.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.942+0000 7fe1c4ff9700 1 -- 172.21.15.12:0/1333581491 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe1c0012030 con 0x7fe1c8158b50 2023-07-08T12:38:39.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.942+0000 7fe1c4ff9700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe1a806ace0 0x7fe1a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:39.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.942+0000 7fe1c4ff9700 1 -- 172.21.15.12:0/1333581491 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe1c008b360 con 0x7fe1c8158b50 2023-07-08T12:38:39.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.943+0000 7fe1c7fff700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe1a806ace0 0x7fe1a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:39.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.943+0000 7fe1c7fff700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe1a806ace0 0x7fe1a806d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0000f60 tx=0x7fe1b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:39.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:39.947+0000 7fe1c4ff9700 1 -- 172.21.15.12:0/1333581491 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe1c0056130 con 0x7fe1c8158b50 2023-07-08T12:38:40.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.146+0000 7fe1cea47700 1 -- 172.21.15.12:0/1333581491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe1c80cc2f0 con 0x7fe1c8158b50 2023-07-08T12:38:40.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.147+0000 7fe1c4ff9700 1 -- 172.21.15.12:0/1333581491 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe1c0059750 con 0x7fe1c8158b50 2023-07-08T12:38:40.150 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:38:40.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.149+0000 7fe1cea47700 1 -- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe1a806ace0 msgr2=0x7fe1a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:40.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.149+0000 7fe1cea47700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe1a806ace0 0x7fe1a806d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0000f60 tx=0x7fe1b0009040 comp rx=0 tx=0).stop 2023-07-08T12:38:40.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.149+0000 7fe1cea47700 1 -- 172.21.15.12:0/1333581491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1c8158b50 msgr2=0x7fe1c815c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:40.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.149+0000 7fe1cea47700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1c8158b50 0x7fe1c815c4c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe1c806caf0 tx=0x7fe1c0004ad0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.149+0000 7fe1cea47700 1 -- 172.21.15.12:0/1333581491 shutdown_connections 2023-07-08T12:38:40.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.149+0000 7fe1cea47700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe1a806ace0 0x7fe1a806d190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.149+0000 7fe1cea47700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1c8158b50 0x7fe1c815c4c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.149+0000 7fe1cea47700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1c8070b00 0x7fe1c80c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.149+0000 7fe1cea47700 1 --2- 172.21.15.12:0/1333581491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1c80cc6d0 0x7fe1c8152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.149+0000 7fe1cea47700 1 -- 172.21.15.12:0/1333581491 >> 172.21.15.12:0/1333581491 conn(0x7fe1c80cfc80 msgr2=0x7fe1c80afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:40.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.149+0000 7fe1cea47700 1 -- 172.21.15.12:0/1333581491 shutdown_connections 2023-07-08T12:38:40.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.149+0000 7fe1cea47700 1 -- 172.21.15.12:0/1333581491 wait complete. 2023-07-08T12:38:40.166 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-08T12:38:40.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.291+0000 7f6baabbe700 1 Processor -- start 2023-07-08T12:38:40.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.291+0000 7f6baabbe700 1 -- start start 2023-07-08T12:38:40.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.292+0000 7f6baabbe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6ba415fad0 0x7f6ba40cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:40.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.292+0000 7f6baabbe700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ba40ce130 0x7f6ba406b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:40.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.292+0000 7f6baabbe700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba415fea0 con 0x7f6ba40cfc30 2023-07-08T12:38:40.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.292+0000 7f6baabbe700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba406e750 con 0x7f6ba415fad0 2023-07-08T12:38:40.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.292+0000 7f6baabbe700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba406ea30 con 0x7f6ba40ce130 2023-07-08T12:38:40.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.292+0000 7f6ba895a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6ba415fad0 0x7f6ba40cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:40.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.292+0000 7f6ba895a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6ba415fad0 0x7f6ba40cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50080/0 (socket says 172.21.15.12:50080) 2023-07-08T12:38:40.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.292+0000 7f6ba895a700 1 -- 172.21.15.12:0/3368582286 learned_addr learned my addr 172.21.15.12:0/3368582286 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:40.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.292+0000 7f6ba3fff700 1 --2- 172.21.15.12:0/3368582286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ba40ce130 0x7f6ba406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:40.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.293+0000 7f6ba895a700 1 -- 172.21.15.12:0/3368582286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ba40ce130 msgr2=0x7f6ba406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:40.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.293+0000 7f6ba895a700 1 --2- 172.21.15.12:0/3368582286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ba40ce130 0x7f6ba406b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.293+0000 7f6ba895a700 1 -- 172.21.15.12:0/3368582286 >> v1:172.21.15.160:6789/0 conn(0x7f6ba40cfc30 legacy=0x7f6ba40ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:40.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.293+0000 7f6ba895a700 1 -- 172.21.15.12:0/3368582286 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba406ed10 con 0x7f6ba415fad0 2023-07-08T12:38:40.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.293+0000 7f6ba895a700 1 --2- 172.21.15.12:0/3368582286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ba415fad0 0x7f6ba40cdbf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098f0 tx=0x7f6b98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359328904a4c6586 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:40.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.293+0000 7f6ba37fe700 1 -- 172.21.15.12:0/3368582286 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9800b8a0 con 0x7f6ba415fad0 2023-07-08T12:38:40.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.293+0000 7f6ba37fe700 1 -- 172.21.15.12:0/3368582286 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9800ba00 con 0x7f6ba415fad0 2023-07-08T12:38:40.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.293+0000 7f6ba37fe700 1 -- 172.21.15.12:0/3368582286 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9800bd90 con 0x7f6ba415fad0 2023-07-08T12:38:40.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.293+0000 7f6baabbe700 1 -- 172.21.15.12:0/3368582286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ba415fad0 msgr2=0x7f6ba40cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:40.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.293+0000 7f6baabbe700 1 --2- 172.21.15.12:0/3368582286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ba415fad0 0x7f6ba40cdbf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098f0 tx=0x7f6b98009c00 comp rx=0 tx=0).stop 2023-07-08T12:38:40.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.294+0000 7f6baabbe700 1 -- 172.21.15.12:0/3368582286 shutdown_connections 2023-07-08T12:38:40.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.294+0000 7f6baabbe700 1 --2- 172.21.15.12:0/3368582286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ba415fad0 0x7f6ba40cdbf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.294+0000 7f6baabbe700 1 --2- 172.21.15.12:0/3368582286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ba40ce130 0x7f6ba406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.294+0000 7f6baabbe700 1 -- 172.21.15.12:0/3368582286 >> 172.21.15.12:0/3368582286 conn(0x7f6ba4099c60 msgr2=0x7f6ba4089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:40.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.294+0000 7f6baabbe700 1 -- 172.21.15.12:0/3368582286 shutdown_connections 2023-07-08T12:38:40.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.294+0000 7f6baabbe700 1 -- 172.21.15.12:0/3368582286 wait complete. 2023-07-08T12:38:40.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.295+0000 7f6baabbe700 1 Processor -- start 2023-07-08T12:38:40.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.295+0000 7f6baabbe700 1 -- start start 2023-07-08T12:38:40.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6baabbe700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ba40ce130 0x7f6ba4154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:40.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6baabbe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ba40cfc30 0x7f6ba4154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:40.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6baabbe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ba415fad0 0x7f6ba40c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:40.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6baabbe700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40c9930 con 0x7f6ba40cfc30 2023-07-08T12:38:40.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6baabbe700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40c8d20 con 0x7f6ba415fad0 2023-07-08T12:38:40.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6baabbe700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40c9000 con 0x7f6ba40ce130 2023-07-08T12:38:40.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6ba3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ba40cfc30 0x7f6ba4154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:40.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6ba3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ba40cfc30 0x7f6ba4154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50088/0 (socket says 172.21.15.12:50088) 2023-07-08T12:38:40.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6ba3fff700 1 -- 172.21.15.12:0/42691193 learned_addr learned my addr 172.21.15.12:0/42691193 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:40.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6ba895a700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ba40ce130 0x7f6ba4154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:40.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6ba915b700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ba415fad0 0x7f6ba40c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:40.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6ba3fff700 1 -- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ba40ce130 msgr2=0x7f6ba4154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:40.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6ba3fff700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ba40ce130 0x7f6ba4154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6ba3fff700 1 -- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ba415fad0 msgr2=0x7f6ba40c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:40.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6ba3fff700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ba415fad0 0x7f6ba40c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6ba3fff700 1 -- 172.21.15.12:0/42691193 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b98009550 con 0x7f6ba40cfc30 2023-07-08T12:38:40.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6ba915b700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ba415fad0 0x7f6ba40c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:40.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6ba895a700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ba40ce130 0x7f6ba4154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:40.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.296+0000 7f6ba3fff700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ba40cfc30 0x7f6ba4154280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000e810 tx=0x7f6b9000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:40.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.297+0000 7f6ba17fa700 1 -- 172.21.15.12:0/42691193 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90013610 con 0x7f6ba40cfc30 2023-07-08T12:38:40.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.297+0000 7f6ba17fa700 1 -- 172.21.15.12:0/42691193 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b90013770 con 0x7f6ba40cfc30 2023-07-08T12:38:40.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.297+0000 7f6baabbe700 1 -- 172.21.15.12:0/42691193 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba40c92e0 con 0x7f6ba40cfc30 2023-07-08T12:38:40.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.297+0000 7f6ba17fa700 1 -- 172.21.15.12:0/42691193 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90013b20 con 0x7f6ba40cfc30 2023-07-08T12:38:40.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.297+0000 7f6baabbe700 1 -- 172.21.15.12:0/42691193 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba40b3290 con 0x7f6ba40cfc30 2023-07-08T12:38:40.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.298+0000 7f6baabbe700 1 -- 172.21.15.12:0/42691193 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba40c9aa0 con 0x7f6ba40cfc30 2023-07-08T12:38:40.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.298+0000 7f6ba17fa700 1 -- 172.21.15.12:0/42691193 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6b90013c80 con 0x7f6ba40cfc30 2023-07-08T12:38:40.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.299+0000 7f6ba17fa700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6b8806ace0 0x7f6b8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:40.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.299+0000 7f6ba17fa700 1 -- 172.21.15.12:0/42691193 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6b90089440 con 0x7f6ba40cfc30 2023-07-08T12:38:40.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.299+0000 7f6ba895a700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6b8806ace0 0x7f6b8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:40.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.299+0000 7f6ba895a700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6b8806ace0 0x7f6b8806d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6b98014040 tx=0x7f6b98008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:40.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.303+0000 7f6ba17fa700 1 -- 172.21.15.12:0/42691193 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6b90054210 con 0x7f6ba40cfc30 2023-07-08T12:38:40.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.501+0000 7f6baabbe700 1 -- 172.21.15.12:0/42691193 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ba404ea60 con 0x7f6ba40cfc30 2023-07-08T12:38:40.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.502+0000 7f6ba17fa700 1 -- 172.21.15.12:0/42691193 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6b90057830 con 0x7f6ba40cfc30 2023-07-08T12:38:40.504 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:38:40.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.504+0000 7f6baabbe700 1 -- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6b8806ace0 msgr2=0x7f6b8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:40.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.504+0000 7f6baabbe700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6b8806ace0 0x7f6b8806d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6b98014040 tx=0x7f6b98008030 comp rx=0 tx=0).stop 2023-07-08T12:38:40.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.504+0000 7f6baabbe700 1 -- 172.21.15.12:0/42691193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ba40cfc30 msgr2=0x7f6ba4154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:40.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.504+0000 7f6baabbe700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ba40cfc30 0x7f6ba4154280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000e810 tx=0x7f6b9000ebd0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.504+0000 7f6baabbe700 1 -- 172.21.15.12:0/42691193 shutdown_connections 2023-07-08T12:38:40.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.504+0000 7f6baabbe700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6b8806ace0 0x7f6b8806d190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.504+0000 7f6baabbe700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ba415fad0 0x7f6ba40c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.504+0000 7f6baabbe700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ba40ce130 0x7f6ba4154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.504+0000 7f6baabbe700 1 --2- 172.21.15.12:0/42691193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ba40cfc30 0x7f6ba4154280 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.504+0000 7f6baabbe700 1 -- 172.21.15.12:0/42691193 >> 172.21.15.12:0/42691193 conn(0x7f6ba4099c60 msgr2=0x7f6ba40af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.504+0000 7f6baabbe700 1 -- 172.21.15.12:0/42691193 shutdown_connections 2023-07-08T12:38:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.504+0000 7f6baabbe700 1 -- 172.21.15.12:0/42691193 wait complete. 2023-07-08T12:38:40.519 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-08T12:38:40.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.647+0000 7fcde8d23700 1 Processor -- start 2023-07-08T12:38:40.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.647+0000 7fcde8d23700 1 -- start start 2023-07-08T12:38:40.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.648+0000 7fcde8d23700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcde4156a10 0x7fcde406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:40.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.648+0000 7fcde8d23700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcde415d390 0x7fcde41608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:40.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.648+0000 7fcde8d23700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcde40cc240 0x7fcde40cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:40.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.648+0000 7fcde8d23700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde40ccc00 con 0x7fcde40cc240 2023-07-08T12:38:40.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.648+0000 7fcde8d23700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde4160410 con 0x7fcde4156a10 2023-07-08T12:38:40.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.648+0000 7fcde8d23700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4156de0 con 0x7fcde415d390 2023-07-08T12:38:40.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.648+0000 7fcde2d9d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcde40cc240 0x7fcde40cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:40.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.648+0000 7fcde2d9d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcde40cc240 0x7fcde40cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50098/0 (socket says 172.21.15.12:50098) 2023-07-08T12:38:40.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.648+0000 7fcde2d9d700 1 -- 172.21.15.12:0/2413281184 learned_addr learned my addr 172.21.15.12:0/2413281184 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:40.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.649+0000 7fcde1d9b700 1 --2- 172.21.15.12:0/2413281184 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcde415d390 0x7fcde41608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:40.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.649+0000 7fcde259c700 1 --2- 172.21.15.12:0/2413281184 >> v2:172.21.15.160:3300/0 conn(0x7fcde4156a10 0x7fcde406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:40.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.649+0000 7fcde2d9d700 1 -- 172.21.15.12:0/2413281184 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcde415d390 msgr2=0x7fcde41608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:40.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.649+0000 7fcde2d9d700 1 --2- 172.21.15.12:0/2413281184 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcde415d390 0x7fcde41608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.649+0000 7fcde2d9d700 1 -- 172.21.15.12:0/2413281184 >> v2:172.21.15.160:3300/0 conn(0x7fcde4156a10 msgr2=0x7fcde406e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:40.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.649+0000 7fcde2d9d700 1 --2- 172.21.15.12:0/2413281184 >> v2:172.21.15.160:3300/0 conn(0x7fcde4156a10 0x7fcde406e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.649+0000 7fcde2d9d700 1 -- 172.21.15.12:0/2413281184 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde406ec70 con 0x7fcde40cc240 2023-07-08T12:38:40.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.649+0000 7fcde2d9d700 1 --2- 172.21.15.12:0/2413281184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde40cc240 0x7fcde40cc6c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcde4072490 tx=0x7fcddc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a394024a85f77e04 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:40.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.649+0000 7fcde159a700 1 -- 172.21.15.12:0/2413281184 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00e070 con 0x7fcde40cc240 2023-07-08T12:38:40.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.649+0000 7fcde159a700 1 -- 172.21.15.12:0/2413281184 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc00b910 con 0x7fcde40cc240 2023-07-08T12:38:40.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.649+0000 7fcde159a700 1 -- 172.21.15.12:0/2413281184 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00bcc0 con 0x7fcde40cc240 2023-07-08T12:38:40.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.650+0000 7fcde8d23700 1 -- 172.21.15.12:0/2413281184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde40cc240 msgr2=0x7fcde40cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:40.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.650+0000 7fcde8d23700 1 --2- 172.21.15.12:0/2413281184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde40cc240 0x7fcde40cc6c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcde4072490 tx=0x7fcddc00b330 comp rx=0 tx=0).stop 2023-07-08T12:38:40.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.650+0000 7fcde8d23700 1 -- 172.21.15.12:0/2413281184 shutdown_connections 2023-07-08T12:38:40.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.650+0000 7fcde8d23700 1 --2- 172.21.15.12:0/2413281184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde40cc240 0x7fcde40cc6c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.650+0000 7fcde8d23700 1 --2- 172.21.15.12:0/2413281184 >> v2:172.21.15.160:3300/0 conn(0x7fcde4156a10 0x7fcde406e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.650+0000 7fcde8d23700 1 --2- 172.21.15.12:0/2413281184 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcde415d390 0x7fcde41608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.650+0000 7fcde8d23700 1 -- 172.21.15.12:0/2413281184 >> 172.21.15.12:0/2413281184 conn(0x7fcde4098b50 msgr2=0x7fcde406b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:40.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.650+0000 7fcde8d23700 1 -- 172.21.15.12:0/2413281184 shutdown_connections 2023-07-08T12:38:40.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.650+0000 7fcde8d23700 1 -- 172.21.15.12:0/2413281184 wait complete. 2023-07-08T12:38:40.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.652+0000 7fcde8d23700 1 Processor -- start 2023-07-08T12:38:40.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.652+0000 7fcde8d23700 1 -- start start 2023-07-08T12:38:40.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.652+0000 7fcde8d23700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcde40cc240 0x7fcde41f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:40.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde8d23700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcde4156a10 0x7fcde40ad470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:40.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde8d23700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde415d390 0x7fcde40ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:40.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde8d23700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40b36a0 con 0x7fcde415d390 2023-07-08T12:38:40.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde8d23700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40b3980 con 0x7fcde40cc240 2023-07-08T12:38:40.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde8d23700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40b3c60 con 0x7fcde4156a10 2023-07-08T12:38:40.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde415d390 0x7fcde40ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:40.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde415d390 0x7fcde40ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50104/0 (socket says 172.21.15.12:50104) 2023-07-08T12:38:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde2d9d700 1 -- 172.21.15.12:0/929676509 learned_addr learned my addr 172.21.15.12:0/929676509 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde1d9b700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcde4156a10 0x7fcde40ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde259c700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcde40cc240 0x7fcde41f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde2d9d700 1 -- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcde4156a10 msgr2=0x7fcde40ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde2d9d700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcde4156a10 0x7fcde40ad470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde2d9d700 1 -- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcde40cc240 msgr2=0x7fcde41f2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde2d9d700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcde40cc240 0x7fcde41f2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde2d9d700 1 -- 172.21.15.12:0/929676509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc009b30 con 0x7fcde415d390 2023-07-08T12:38:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde2d9d700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde415d390 0x7fcde40ad9b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcde4070aa0 tx=0x7fcddc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcdd2ffd700 1 -- 172.21.15.12:0/929676509 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00ce80 con 0x7fcde415d390 2023-07-08T12:38:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcdd2ffd700 1 -- 172.21.15.12:0/929676509 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc004250 con 0x7fcde415d390 2023-07-08T12:38:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcdd2ffd700 1 -- 172.21.15.12:0/929676509 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc004600 con 0x7fcde415d390 2023-07-08T12:38:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde8d23700 1 -- 172.21.15.12:0/929676509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde40b3f40 con 0x7fcde415d390 2023-07-08T12:38:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.653+0000 7fcde8d23700 1 -- 172.21.15.12:0/929676509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde40b4220 con 0x7fcde415d390 2023-07-08T12:38:40.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.654+0000 7fcde8d23700 1 -- 172.21.15.12:0/929676509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde41f3240 con 0x7fcde415d390 2023-07-08T12:38:40.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.655+0000 7fcdd2ffd700 1 -- 172.21.15.12:0/929676509 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcddc004760 con 0x7fcde415d390 2023-07-08T12:38:40.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.655+0000 7fcdd2ffd700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcdc406ace0 0x7fcdc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.656+0000 7fcdd2ffd700 1 -- 172.21.15.12:0/929676509 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcddc08ac00 con 0x7fcde415d390 2023-07-08T12:38:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.656+0000 7fcde259c700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcdc406ace0 0x7fcdc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.656+0000 7fcde259c700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcdc406ace0 0x7fcdc406d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009ad0 tx=0x7fcdd8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.660+0000 7fcdd2ffd700 1 -- 172.21.15.12:0/929676509 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcde41f3240 con 0x7fcde415d390 2023-07-08T12:38:40.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.860+0000 7fcde8d23700 1 -- 172.21.15.12:0/929676509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcde41f3240 con 0x7fcde415d390 2023-07-08T12:38:40.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.860+0000 7fcdd2ffd700 1 -- 172.21.15.12:0/929676509 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcde41f3240 con 0x7fcde415d390 2023-07-08T12:38:40.863 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-07-08T12:38:40.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.862+0000 7fcde8d23700 1 -- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcdc406ace0 msgr2=0x7fcdc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:40.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.862+0000 7fcde8d23700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcdc406ace0 0x7fcdc406d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009ad0 tx=0x7fcdd8008040 comp rx=0 tx=0).stop 2023-07-08T12:38:40.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.862+0000 7fcde8d23700 1 -- 172.21.15.12:0/929676509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde415d390 msgr2=0x7fcde40ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:40.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.862+0000 7fcde8d23700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde415d390 0x7fcde40ad9b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcde4070aa0 tx=0x7fcddc004ad0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.862+0000 7fcde8d23700 1 -- 172.21.15.12:0/929676509 shutdown_connections 2023-07-08T12:38:40.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.862+0000 7fcde8d23700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcdc406ace0 0x7fcdc406d190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.862+0000 7fcde8d23700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde415d390 0x7fcde40ad9b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.862+0000 7fcde8d23700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcde40cc240 0x7fcde41f2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.862+0000 7fcde8d23700 1 --2- 172.21.15.12:0/929676509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcde4156a10 0x7fcde40ad470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:40.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.862+0000 7fcde8d23700 1 -- 172.21.15.12:0/929676509 >> 172.21.15.12:0/929676509 conn(0x7fcde4098b50 msgr2=0x7fcde40ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:40.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.862+0000 7fcde8d23700 1 -- 172.21.15.12:0/929676509 shutdown_connections 2023-07-08T12:38:40.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:40.862+0000 7fcde8d23700 1 -- 172.21.15.12:0/929676509 wait complete. 2023-07-08T12:38:40.877 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-08T12:38:41.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.004+0000 7f3ea2745700 1 Processor -- start 2023-07-08T12:38:41.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.004+0000 7f3ea2745700 1 -- start start 2023-07-08T12:38:41.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3ea2745700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e9c156a10 0x7f3e9c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3ea2745700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c160410 con 0x7f3e9c0cc240 2023-07-08T12:38:41.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3ea2745700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c156de0 con 0x7f3e9c15d390 2023-07-08T12:38:41.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3ea2745700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c06ec70 con 0x7f3e9c156a10 2023-07-08T12:38:41.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3ea0ce2700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3e9c0cc240 0x7f3e9c0ae440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37508/0 (socket says 172.21.15.12:37508) 2023-07-08T12:38:41.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3ea0ce2700 1 -- 172.21.15.12:0/4202890300 learned_addr learned my addr 172.21.15.12:0/4202890300 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:41.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3e9bfff700 1 --2- 172.21.15.12:0/4202890300 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e9c156a10 0x7f3e9c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3e9affd700 1 -- 172.21.15.12:0/4202890300 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539828916 0 0) 0x7f3e9c160410 con 0x7f3e9c0cc240 2023-07-08T12:38:41.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3e9affd700 1 -- 172.21.15.12:0/4202890300 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e840035b0 con 0x7f3e9c0cc240 2023-07-08T12:38:41.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3e9affd700 1 -- 172.21.15.12:0/4202890300 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611392556 0 0) 0x7f3e9c156de0 con 0x7f3e9c15d390 2023-07-08T12:38:41.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3e9affd700 1 -- 172.21.15.12:0/4202890300 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e9c160410 con 0x7f3e9c15d390 2023-07-08T12:38:41.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3e9bfff700 1 -- 172.21.15.12:0/4202890300 >> v1:172.21.15.160:6789/0 conn(0x7f3e9c15d390 legacy=0x7f3e9c1608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3e9bfff700 1 -- 172.21.15.12:0/4202890300 >> v1:172.21.15.12:6789/0 conn(0x7f3e9c0cc240 legacy=0x7f3e9c0ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.005+0000 7f3e9bfff700 1 -- 172.21.15.12:0/4202890300 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c06e6b0 con 0x7f3e9c156a10 2023-07-08T12:38:41.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.011+0000 7f3e9bfff700 1 --2- 172.21.15.12:0/4202890300 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e9c156a10 0x7f3e9c06e0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009850 tx=0x7f3e8c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b935d08f134a4181 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:41.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.011+0000 7f3e9affd700 1 -- 172.21.15.12:0/4202890300 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c005e30 con 0x7f3e9c156a10 2023-07-08T12:38:41.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.011+0000 7f3e9affd700 1 -- 172.21.15.12:0/4202890300 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c006030 con 0x7f3e9c156a10 2023-07-08T12:38:41.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.011+0000 7f3e9affd700 1 -- 172.21.15.12:0/4202890300 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0063e0 con 0x7f3e9c156a10 2023-07-08T12:38:41.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.012+0000 7f3ea2745700 1 -- 172.21.15.12:0/4202890300 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e9c156a10 msgr2=0x7f3e9c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.012+0000 7f3ea2745700 1 --2- 172.21.15.12:0/4202890300 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e9c156a10 0x7f3e9c06e0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009850 tx=0x7f3e8c009b60 comp rx=0 tx=0).stop 2023-07-08T12:38:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.012+0000 7f3ea2745700 1 -- 172.21.15.12:0/4202890300 shutdown_connections 2023-07-08T12:38:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.012+0000 7f3ea2745700 1 --2- 172.21.15.12:0/4202890300 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e9c156a10 0x7f3e9c06e0c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.012+0000 7f3ea2745700 1 -- 172.21.15.12:0/4202890300 >> 172.21.15.12:0/4202890300 conn(0x7f3e9c098b50 msgr2=0x7f3e9c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.012+0000 7f3ea2745700 1 -- 172.21.15.12:0/4202890300 shutdown_connections 2023-07-08T12:38:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.012+0000 7f3ea2745700 1 -- 172.21.15.12:0/4202890300 wait complete. 2023-07-08T12:38:41.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.014+0000 7f3ea2745700 1 Processor -- start 2023-07-08T12:38:41.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.014+0000 7f3ea2745700 1 -- start start 2023-07-08T12:38:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.015+0000 7f3ea2745700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9c0cc240 0x7f3e9c0b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.015+0000 7f3ea2745700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e9c156a10 0x7f3e9c06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.015+0000 7f3ea2745700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e9c15d390 0x7f3e9c06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.015+0000 7f3ea2745700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c155da0 con 0x7f3e9c0cc240 2023-07-08T12:38:41.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.015+0000 7f3ea2745700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c0b5560 con 0x7f3e9c156a10 2023-07-08T12:38:41.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.015+0000 7f3ea2745700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c0b5280 con 0x7f3e9c15d390 2023-07-08T12:38:41.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.015+0000 7f3e9bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9c0cc240 0x7f3e9c0b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.015+0000 7f3e9bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9c0cc240 0x7f3e9c0b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50114/0 (socket says 172.21.15.12:50114) 2023-07-08T12:38:41.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.015+0000 7f3e9bfff700 1 -- 172.21.15.12:0/4231018066 learned_addr learned my addr 172.21.15.12:0/4231018066 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:41.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.015+0000 7f3ea0ce2700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e9c15d390 0x7f3e9c06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.016+0000 7f3e9b7fe700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e9c156a10 0x7f3e9c06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.016+0000 7f3e9bfff700 1 -- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e9c15d390 msgr2=0x7f3e9c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.016+0000 7f3e9bfff700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e9c15d390 0x7f3e9c06dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.016+0000 7f3e9bfff700 1 -- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e9c156a10 msgr2=0x7f3e9c06d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.016+0000 7f3e9bfff700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e9c156a10 0x7f3e9c06d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.016+0000 7f3e9bfff700 1 -- 172.21.15.12:0/4231018066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c009430 con 0x7f3e9c0cc240 2023-07-08T12:38:41.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.016+0000 7f3e9bfff700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9c0cc240 0x7f3e9c0b2c90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c000c00 tx=0x7f3e8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.016+0000 7f3e98ff9700 1 -- 172.21.15.12:0/4231018066 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c006790 con 0x7f3e9c0cc240 2023-07-08T12:38:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.016+0000 7f3e98ff9700 1 -- 172.21.15.12:0/4231018066 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c020070 con 0x7f3e9c0cc240 2023-07-08T12:38:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.016+0000 7f3ea2745700 1 -- 172.21.15.12:0/4231018066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c1e5e90 con 0x7f3e9c0cc240 2023-07-08T12:38:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.016+0000 7f3e98ff9700 1 -- 172.21.15.12:0/4231018066 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c025670 con 0x7f3e9c0cc240 2023-07-08T12:38:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.016+0000 7f3ea2745700 1 -- 172.21.15.12:0/4231018066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c1e6450 con 0x7f3e9c0cc240 2023-07-08T12:38:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.017+0000 7f3ea2745700 1 -- 172.21.15.12:0/4231018066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c066af0 con 0x7f3e9c0cc240 2023-07-08T12:38:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.018+0000 7f3e98ff9700 1 -- 172.21.15.12:0/4231018066 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e8c022030 con 0x7f3e9c0cc240 2023-07-08T12:38:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.018+0000 7f3e98ff9700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e8806ace0 0x7f3e8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.018+0000 7f3e98ff9700 1 -- 172.21.15.12:0/4231018066 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3e8c08aa50 con 0x7f3e9c0cc240 2023-07-08T12:38:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.019+0000 7f3e9b7fe700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e8806ace0 0x7f3e8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.019+0000 7f3e9b7fe700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e8806ace0 0x7f3e8806d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0ae860 tx=0x7f3e90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:41.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.023+0000 7f3e98ff9700 1 -- 172.21.15.12:0/4231018066 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3e8c0558a0 con 0x7f3e9c0cc240 2023-07-08T12:38:41.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.219+0000 7f3ea2745700 1 -- 172.21.15.12:0/4231018066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3e9c04ea60 con 0x7f3e9c0cc240 2023-07-08T12:38:41.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.219+0000 7f3e98ff9700 1 -- 172.21.15.12:0/4231018066 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3e8c058ec0 con 0x7f3e9c0cc240 2023-07-08T12:38:41.222 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:38:41.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.221+0000 7f3ea2745700 1 -- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e8806ace0 msgr2=0x7f3e8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.221+0000 7f3ea2745700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e8806ace0 0x7f3e8806d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0ae860 tx=0x7f3e90009040 comp rx=0 tx=0).stop 2023-07-08T12:38:41.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.221+0000 7f3ea2745700 1 -- 172.21.15.12:0/4231018066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9c0cc240 msgr2=0x7f3e9c0b2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.221+0000 7f3ea2745700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9c0cc240 0x7f3e9c0b2c90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c000c00 tx=0x7f3e8c004750 comp rx=0 tx=0).stop 2023-07-08T12:38:41.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.221+0000 7f3ea2745700 1 -- 172.21.15.12:0/4231018066 shutdown_connections 2023-07-08T12:38:41.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.221+0000 7f3ea2745700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3e8806ace0 0x7f3e8806d190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.221+0000 7f3ea2745700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e9c15d390 0x7f3e9c06dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.221+0000 7f3ea2745700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9c0cc240 0x7f3e9c0b2c90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.221+0000 7f3ea2745700 1 --2- 172.21.15.12:0/4231018066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e9c156a10 0x7f3e9c06d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.221+0000 7f3ea2745700 1 -- 172.21.15.12:0/4231018066 >> 172.21.15.12:0/4231018066 conn(0x7f3e9c098b50 msgr2=0x7f3e9c0cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:41.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.221+0000 7f3ea2745700 1 -- 172.21.15.12:0/4231018066 shutdown_connections 2023-07-08T12:38:41.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.221+0000 7f3ea2745700 1 -- 172.21.15.12:0/4231018066 wait complete. 2023-07-08T12:38:41.236 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-08T12:38:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.359+0000 7fd691e74700 1 Processor -- start 2023-07-08T12:38:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.359+0000 7fd691e74700 1 -- start start 2023-07-08T12:38:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.360+0000 7fd691e74700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd68c0cd7a0 0x7fd68c0cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.360+0000 7fd691e74700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd68c0caf90 0x7fd68c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.360+0000 7fd691e74700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd68c1594b0 con 0x7fd68c0ccff0 2023-07-08T12:38:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.360+0000 7fd691e74700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd68c0ce0b0 con 0x7fd68c0cd7a0 2023-07-08T12:38:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.360+0000 7fd691e74700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c0ca1d0 con 0x7fd68c0caf90 2023-07-08T12:38:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.360+0000 7fd68bfff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd68c0ccff0 0x7fd68c069f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37522/0 (socket says 172.21.15.12:37522) 2023-07-08T12:38:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.360+0000 7fd68bfff700 1 -- 172.21.15.12:0/2940118801 learned_addr learned my addr 172.21.15.12:0/2940118801 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.360+0000 7fd68b7fe700 1 --2- 172.21.15.12:0/2940118801 >> v2:172.21.15.160:3300/0 conn(0x7fd68c0cd7a0 0x7fd68c0cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.360+0000 7fd68affd700 1 --2- 172.21.15.12:0/2940118801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd68c0caf90 0x7fd68c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.360+0000 7fd68b7fe700 1 -- 172.21.15.12:0/2940118801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd68c0caf90 msgr2=0x7fd68c0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.360+0000 7fd68b7fe700 1 --2- 172.21.15.12:0/2940118801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd68c0caf90 0x7fd68c0ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.360+0000 7fd68b7fe700 1 -- 172.21.15.12:0/2940118801 >> v1:172.21.15.12:6789/0 conn(0x7fd68c0ccff0 legacy=0x7fd68c069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.360+0000 7fd68b7fe700 1 -- 172.21.15.12:0/2940118801 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c0ca4b0 con 0x7fd68c0cd7a0 2023-07-08T12:38:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.361+0000 7fd68b7fe700 1 --2- 172.21.15.12:0/2940118801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd68c0cd7a0 0x7fd68c0cdb70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009870 tx=0x7fd67c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4da510c9e94f8ea2 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:41.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.362+0000 7fd68a7fc700 1 -- 172.21.15.12:0/2940118801 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c011830 con 0x7fd68c0cd7a0 2023-07-08T12:38:41.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.363+0000 7fd68a7fc700 1 -- 172.21.15.12:0/2940118801 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67c011990 con 0x7fd68c0cd7a0 2023-07-08T12:38:41.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.363+0000 7fd68a7fc700 1 -- 172.21.15.12:0/2940118801 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c011d20 con 0x7fd68c0cd7a0 2023-07-08T12:38:41.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.363+0000 7fd691e74700 1 -- 172.21.15.12:0/2940118801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd68c0cd7a0 msgr2=0x7fd68c0cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.363+0000 7fd691e74700 1 --2- 172.21.15.12:0/2940118801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd68c0cd7a0 0x7fd68c0cdb70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009870 tx=0x7fd67c009b80 comp rx=0 tx=0).stop 2023-07-08T12:38:41.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.363+0000 7fd691e74700 1 -- 172.21.15.12:0/2940118801 shutdown_connections 2023-07-08T12:38:41.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.363+0000 7fd691e74700 1 --2- 172.21.15.12:0/2940118801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd68c0cd7a0 0x7fd68c0cdb70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.363+0000 7fd691e74700 1 --2- 172.21.15.12:0/2940118801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd68c0caf90 0x7fd68c0ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.363+0000 7fd691e74700 1 -- 172.21.15.12:0/2940118801 >> 172.21.15.12:0/2940118801 conn(0x7fd68c0cb9f0 msgr2=0x7fd68c0cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.363+0000 7fd691e74700 1 -- 172.21.15.12:0/2940118801 shutdown_connections 2023-07-08T12:38:41.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.363+0000 7fd691e74700 1 -- 172.21.15.12:0/2940118801 wait complete. 2023-07-08T12:38:41.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.364+0000 7fd691e74700 1 Processor -- start 2023-07-08T12:38:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.365+0000 7fd691e74700 1 -- start start 2023-07-08T12:38:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.365+0000 7fd691e74700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd68c0caf90 0x7fd68c152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.365+0000 7fd691e74700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd68c0ccff0 0x7fd68c1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.365+0000 7fd691e74700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd68c0cd7a0 0x7fd68c150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.365+0000 7fd691e74700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c0c3420 con 0x7fd68c0caf90 2023-07-08T12:38:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.365+0000 7fd691e74700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c0c3700 con 0x7fd68c0cd7a0 2023-07-08T12:38:41.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.365+0000 7fd691e74700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c0c39e0 con 0x7fd68c0ccff0 2023-07-08T12:38:41.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.365+0000 7fd68b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd68c0caf90 0x7fd68c152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd68b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd68c0caf90 0x7fd68c152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50128/0 (socket says 172.21.15.12:50128) 2023-07-08T12:38:41.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd68b7fe700 1 -- 172.21.15.12:0/3032052996 learned_addr learned my addr 172.21.15.12:0/3032052996 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:41.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd68bfff700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd68c0cd7a0 0x7fd68c150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd68affd700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd68c0ccff0 0x7fd68c1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd68b7fe700 1 -- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd68c0ccff0 msgr2=0x7fd68c1556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd68b7fe700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd68c0ccff0 0x7fd68c1556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd68b7fe700 1 -- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd68c0cd7a0 msgr2=0x7fd68c150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd68b7fe700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd68c0cd7a0 0x7fd68c150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd68b7fe700 1 -- 172.21.15.12:0/3032052996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67c0094a0 con 0x7fd68c0caf90 2023-07-08T12:38:41.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd68affd700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd68c0ccff0 0x7fd68c1556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:41.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd68bfff700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd68c0cd7a0 0x7fd68c150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:41.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd68b7fe700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd68c0caf90 0x7fd68c152f90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd67c014040 tx=0x7fd67c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:41.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd673fff700 1 -- 172.21.15.12:0/3032052996 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0038a0 con 0x7fd68c0caf90 2023-07-08T12:38:41.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd673fff700 1 -- 172.21.15.12:0/3032052996 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67c01e070 con 0x7fd68c0caf90 2023-07-08T12:38:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd691e74700 1 -- 172.21.15.12:0/3032052996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68c0c3cc0 con 0x7fd68c0caf90 2023-07-08T12:38:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd673fff700 1 -- 172.21.15.12:0/3032052996 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c003c20 con 0x7fd68c0caf90 2023-07-08T12:38:41.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.366+0000 7fd691e74700 1 -- 172.21.15.12:0/3032052996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68c1534d0 con 0x7fd68c0caf90 2023-07-08T12:38:41.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.367+0000 7fd691e74700 1 -- 172.21.15.12:0/3032052996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd68c1536c0 con 0x7fd68c0caf90 2023-07-08T12:38:41.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.368+0000 7fd673fff700 1 -- 172.21.15.12:0/3032052996 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd67c02d450 con 0x7fd68c0caf90 2023-07-08T12:38:41.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.368+0000 7fd673fff700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd67806ace0 0x7fd67806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.368+0000 7fd673fff700 1 -- 172.21.15.12:0/3032052996 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd67c08a730 con 0x7fd68c0caf90 2023-07-08T12:38:41.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.368+0000 7fd68affd700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd67806ace0 0x7fd67806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.369+0000 7fd68affd700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd67806ace0 0x7fd67806d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0c2960 tx=0x7fd680008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:41.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.372+0000 7fd673fff700 1 -- 172.21.15.12:0/3032052996 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd67c055580 con 0x7fd68c0caf90 2023-07-08T12:38:41.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.613+0000 7fd691e74700 1 -- 172.21.15.12:0/3032052996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd68c04ea60 con 0x7fd68c0caf90 2023-07-08T12:38:41.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.613+0000 7fd673fff700 1 -- 172.21.15.12:0/3032052996 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fd67c058ba0 con 0x7fd68c0caf90 2023-07-08T12:38:41.617 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:41.617 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3238006310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3238006310},{"type":"v1","addr":"172.21.15.160:6836","nonce":3238006310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3864661454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3864661454},{"type":"v1","addr":"172.21.15.160:6837","nonce":3864661454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6834/946118830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":946118830},{"type":"v1","addr":"172.21.15.12:6834","nonce":946118830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-08T12:38:07.943202+0000","modified":"2023-07-08T12:38:32.319568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6835/3301134992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3301134992},{"type":"v1","addr":"172.21.15.12:6835","nonce":3301134992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-08T12:38:41.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.615+0000 7fd691e74700 1 -- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd67806ace0 msgr2=0x7fd67806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.615+0000 7fd691e74700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd67806ace0 0x7fd67806d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0c2960 tx=0x7fd680008040 comp rx=0 tx=0).stop 2023-07-08T12:38:41.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.615+0000 7fd691e74700 1 -- 172.21.15.12:0/3032052996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd68c0caf90 msgr2=0x7fd68c152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.615+0000 7fd691e74700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd68c0caf90 0x7fd68c152f90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd67c014040 tx=0x7fd67c004750 comp rx=0 tx=0).stop 2023-07-08T12:38:41.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.615+0000 7fd691e74700 1 -- 172.21.15.12:0/3032052996 shutdown_connections 2023-07-08T12:38:41.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.615+0000 7fd691e74700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd67806ace0 0x7fd67806d190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.615+0000 7fd691e74700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd68c0cd7a0 0x7fd68c150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.615+0000 7fd691e74700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd68c0caf90 0x7fd68c152f90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.615+0000 7fd691e74700 1 --2- 172.21.15.12:0/3032052996 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd68c0ccff0 0x7fd68c1556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.615+0000 7fd691e74700 1 -- 172.21.15.12:0/3032052996 >> 172.21.15.12:0/3032052996 conn(0x7fd68c0cb9f0 msgr2=0x7fd68c158750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:41.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.616+0000 7fd691e74700 1 -- 172.21.15.12:0/3032052996 shutdown_connections 2023-07-08T12:38:41.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.616+0000 7fd691e74700 1 -- 172.21.15.12:0/3032052996 wait complete. 2023-07-08T12:38:41.620 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2023-07-08T12:38:41.630 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.752+0000 7f040e4a6700 1 Processor -- start 2023-07-08T12:38:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.752+0000 7f040e4a6700 1 -- start start 2023-07-08T12:38:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f040e4a6700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f040815a370 0x7f040815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f040e4a6700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0408069f30 0x7f040806a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f040e4a6700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f040806cd50 0x7f040806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f040e4a6700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040815a740 con 0x7f0408069f30 2023-07-08T12:38:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f040e4a6700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040806a8e0 con 0x7f040815a370 2023-07-08T12:38:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f040e4a6700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04080b1de0 con 0x7f040806cd50 2023-07-08T12:38:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f04077fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0408069f30 0x7f040806a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f04077fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0408069f30 0x7f040806a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50138/0 (socket says 172.21.15.12:50138) 2023-07-08T12:38:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f04077fe700 1 -- 172.21.15.12:0/1227108285 learned_addr learned my addr 172.21.15.12:0/1227108285 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f040ca43700 1 --2- 172.21.15.12:0/1227108285 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f040806cd50 0x7f040806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f0407fff700 1 --2- 172.21.15.12:0/1227108285 >> v2:172.21.15.160:3300/0 conn(0x7f040815a370 0x7f040815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f04077fe700 1 -- 172.21.15.12:0/1227108285 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f040806cd50 msgr2=0x7f040806d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:41.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f04077fe700 1 --2- 172.21.15.12:0/1227108285 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f040806cd50 0x7f040806d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f04077fe700 1 -- 172.21.15.12:0/1227108285 >> v2:172.21.15.160:3300/0 conn(0x7f040815a370 msgr2=0x7f040815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f04077fe700 1 --2- 172.21.15.12:0/1227108285 >> v2:172.21.15.160:3300/0 conn(0x7f040815a370 0x7f040815e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f04077fe700 1 -- 172.21.15.12:0/1227108285 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040815ed90 con 0x7f0408069f30 2023-07-08T12:38:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.753+0000 7f0407fff700 1 --2- 172.21.15.12:0/1227108285 >> v2:172.21.15.160:3300/0 conn(0x7f040815a370 0x7f040815e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.754+0000 7f04077fe700 1 --2- 172.21.15.12:0/1227108285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0408069f30 0x7f040806a3a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f040806e380 tx=0x7f03f8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba1ee2019aa375d server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.754+0000 7f0406ffd700 1 -- 172.21.15.12:0/1227108285 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f800b9e0 con 0x7f0408069f30 2023-07-08T12:38:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.754+0000 7f0406ffd700 1 -- 172.21.15.12:0/1227108285 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f800bb40 con 0x7f0408069f30 2023-07-08T12:38:41.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.754+0000 7f0406ffd700 1 -- 172.21.15.12:0/1227108285 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f8009e70 con 0x7f0408069f30 2023-07-08T12:38:41.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.754+0000 7f040e4a6700 1 -- 172.21.15.12:0/1227108285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0408069f30 msgr2=0x7f040806a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.754+0000 7f040e4a6700 1 --2- 172.21.15.12:0/1227108285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0408069f30 0x7f040806a3a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f040806e380 tx=0x7f03f8009c30 comp rx=0 tx=0).stop 2023-07-08T12:38:41.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.754+0000 7f040e4a6700 1 -- 172.21.15.12:0/1227108285 shutdown_connections 2023-07-08T12:38:41.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.754+0000 7f040e4a6700 1 --2- 172.21.15.12:0/1227108285 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f040806cd50 0x7f040806d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.754+0000 7f040e4a6700 1 --2- 172.21.15.12:0/1227108285 >> v2:172.21.15.160:3300/0 conn(0x7f040815a370 0x7f040815e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.754+0000 7f040e4a6700 1 --2- 172.21.15.12:0/1227108285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0408069f30 0x7f040806a3a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.754+0000 7f040e4a6700 1 -- 172.21.15.12:0/1227108285 >> 172.21.15.12:0/1227108285 conn(0x7f04080cb9f0 msgr2=0x7f04080cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.754+0000 7f040e4a6700 1 -- 172.21.15.12:0/1227108285 shutdown_connections 2023-07-08T12:38:41.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.754+0000 7f040e4a6700 1 -- 172.21.15.12:0/1227108285 wait complete. 2023-07-08T12:38:41.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.756+0000 7f040e4a6700 1 Processor -- start 2023-07-08T12:38:41.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.756+0000 7f040e4a6700 1 -- start start 2023-07-08T12:38:41.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f040e4a6700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0408069f30 0x7f04080aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f040e4a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f040806cd50 0x7f04080ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f040e4a6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f040815a370 0x7f04081532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f040e4a6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0408152e40 con 0x7f040806cd50 2023-07-08T12:38:41.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f040e4a6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04080ac5a0 con 0x7f040815a370 2023-07-08T12:38:41.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f040e4a6700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0408152510 con 0x7f0408069f30 2023-07-08T12:38:41.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f04077fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f040806cd50 0x7f04080ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f04077fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f040806cd50 0x7f04080ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50144/0 (socket says 172.21.15.12:50144) 2023-07-08T12:38:41.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f04077fe700 1 -- 172.21.15.12:0/3172145064 learned_addr learned my addr 172.21.15.12:0/3172145064 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:41.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f040ca43700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f040815a370 0x7f04081532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f0407fff700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0408069f30 0x7f04080aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f04077fe700 1 -- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0408069f30 msgr2=0x7f04080aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f04077fe700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0408069f30 0x7f04080aca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f04077fe700 1 -- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f040815a370 msgr2=0x7f04081532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f04077fe700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f040815a370 0x7f04081532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f04077fe700 1 -- 172.21.15.12:0/3172145064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f8009550 con 0x7f040806cd50 2023-07-08T12:38:41.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.757+0000 7f040ca43700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f040815a370 0x7f04081532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:41.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.758+0000 7f04077fe700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f040806cd50 0x7f04080ccab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f03f8014040 tx=0x7f03f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:41.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.758+0000 7f0404ff9700 1 -- 172.21.15.12:0/3172145064 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f80036a0 con 0x7f040806cd50 2023-07-08T12:38:41.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.758+0000 7f0404ff9700 1 -- 172.21.15.12:0/3172145064 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f801e070 con 0x7f040806cd50 2023-07-08T12:38:41.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.758+0000 7f0404ff9700 1 -- 172.21.15.12:0/3172145064 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f8003a40 con 0x7f040806cd50 2023-07-08T12:38:41.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.758+0000 7f040e4a6700 1 -- 172.21.15.12:0/3172145064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04081527f0 con 0x7f040806cd50 2023-07-08T12:38:41.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.758+0000 7f040e4a6700 1 -- 172.21.15.12:0/3172145064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04080acf80 con 0x7f040806cd50 2023-07-08T12:38:41.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.759+0000 7f040e4a6700 1 -- 172.21.15.12:0/3172145064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04080ad110 con 0x7f040806cd50 2023-07-08T12:38:41.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.759+0000 7f0404ff9700 1 -- 172.21.15.12:0/3172145064 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f03f8003ba0 con 0x7f040806cd50 2023-07-08T12:38:41.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.760+0000 7f0404ff9700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f03f406ace0 0x7f03f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:41.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.760+0000 7f0404ff9700 1 -- 172.21.15.12:0/3172145064 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f03f8020070 con 0x7f040806cd50 2023-07-08T12:38:41.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.760+0000 7f0407fff700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f03f406ace0 0x7f03f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:41.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.760+0000 7f0407fff700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f03f406ace0 0x7f03f406d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f03fc006fd0 tx=0x7f03fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:41.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.764+0000 7f0404ff9700 1 -- 172.21.15.12:0/3172145064 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f03f8055ad0 con 0x7f040806cd50 2023-07-08T12:38:41.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.962+0000 7f040e4a6700 1 -- 172.21.15.12:0/3172145064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f040804ea60 con 0x7f040806cd50 2023-07-08T12:38:41.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.964+0000 7f0404ff9700 1 -- 172.21.15.12:0/3172145064 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20108 (secure 0 0 0) 0x7f03f8015090 con 0x7f040806cd50 2023-07-08T12:38:41.966 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:41.967 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:12.993803+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:41.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.966+0000 7f040e4a6700 1 -- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f03f406ace0 msgr2=0x7f03f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.966+0000 7f040e4a6700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f03f406ace0 0x7f03f406d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f03fc006fd0 tx=0x7f03fc008040 comp rx=0 tx=0).stop 2023-07-08T12:38:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.966+0000 7f040e4a6700 1 -- 172.21.15.12:0/3172145064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f040806cd50 msgr2=0x7f04080ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.966+0000 7f040e4a6700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f040806cd50 0x7f04080ccab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f03f8014040 tx=0x7f03f8004750 comp rx=0 tx=0).stop 2023-07-08T12:38:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.966+0000 7f040e4a6700 1 -- 172.21.15.12:0/3172145064 shutdown_connections 2023-07-08T12:38:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.966+0000 7f040e4a6700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f03f406ace0 0x7f03f406d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.966+0000 7f040e4a6700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f040815a370 0x7f04081532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.966+0000 7f040e4a6700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0408069f30 0x7f04080aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.966+0000 7f040e4a6700 1 --2- 172.21.15.12:0/3172145064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f040806cd50 0x7f04080ccab0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:41.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.966+0000 7f040e4a6700 1 -- 172.21.15.12:0/3172145064 >> 172.21.15.12:0/3172145064 conn(0x7f04080cb9f0 msgr2=0x7f0408157b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:41.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.966+0000 7f040e4a6700 1 -- 172.21.15.12:0/3172145064 shutdown_connections 2023-07-08T12:38:41.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:41.966+0000 7f040e4a6700 1 -- 172.21.15.12:0/3172145064 wait complete. 2023-07-08T12:38:41.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-08T12:38:05.660194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-08T12:38:06.920177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-08T12:38:10.592749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-08T12:38:41.982 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:38:42.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.107+0000 7efde08e4700 1 Processor -- start 2023-07-08T12:38:42.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.107+0000 7efde08e4700 1 -- start start 2023-07-08T12:38:42.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efde08e4700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc156a10 0x7efddc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efde08e4700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efddc0cc240 0x7efddc0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efde08e4700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efddc160410 con 0x7efddc15d390 2023-07-08T12:38:42.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efde08e4700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc0ccc00 con 0x7efddc0cc240 2023-07-08T12:38:42.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efde08e4700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc156de0 con 0x7efddc156a10 2023-07-08T12:38:42.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efddad9d700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efddc0cc240 0x7efddc0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efdda59c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc156a10 0x7efddc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efdd9d9b700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7efddc15d390 0x7efddc1608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:46768/0 (socket says 172.21.15.12:46768) 2023-07-08T12:38:42.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efdd9d9b700 1 -- 172.21.15.12:0/714417725 learned_addr learned my addr 172.21.15.12:0/714417725 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:42.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efddad9d700 1 -- 172.21.15.12:0/714417725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc156a10 msgr2=0x7efddc06e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:42.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efddad9d700 1 --2- 172.21.15.12:0/714417725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc156a10 0x7efddc06e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efddad9d700 1 -- 172.21.15.12:0/714417725 >> v1:172.21.15.160:6789/0 conn(0x7efddc15d390 legacy=0x7efddc1608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efdd959a700 1 -- 172.21.15.12:0/714417725 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048693772 0 0) 0x7efddc160410 con 0x7efddc15d390 2023-07-08T12:38:42.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.108+0000 7efddad9d700 1 -- 172.21.15.12:0/714417725 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc06ec70 con 0x7efddc0cc240 2023-07-08T12:38:42.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.109+0000 7efddad9d700 1 --2- 172.21.15.12:0/714417725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc0cc240 0x7efddc0cc6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009d90 tx=0x7efdd400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df17ddffb383a154 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:42.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.109+0000 7efdd959a700 1 -- 172.21.15.12:0/714417725 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd400e070 con 0x7efddc0cc240 2023-07-08T12:38:42.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.109+0000 7efdd959a700 1 -- 172.21.15.12:0/714417725 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd400bb20 con 0x7efddc0cc240 2023-07-08T12:38:42.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.109+0000 7efdd959a700 1 -- 172.21.15.12:0/714417725 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd4012430 con 0x7efddc0cc240 2023-07-08T12:38:42.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.109+0000 7efde08e4700 1 -- 172.21.15.12:0/714417725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc0cc240 msgr2=0x7efddc0cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.109+0000 7efde08e4700 1 --2- 172.21.15.12:0/714417725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc0cc240 0x7efddc0cc6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009d90 tx=0x7efdd400b330 comp rx=0 tx=0).stop 2023-07-08T12:38:42.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.109+0000 7efde08e4700 1 -- 172.21.15.12:0/714417725 shutdown_connections 2023-07-08T12:38:42.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.109+0000 7efde08e4700 1 --2- 172.21.15.12:0/714417725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc0cc240 0x7efddc0cc6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.109+0000 7efde08e4700 1 --2- 172.21.15.12:0/714417725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc156a10 0x7efddc06e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.109+0000 7efde08e4700 1 -- 172.21.15.12:0/714417725 >> 172.21.15.12:0/714417725 conn(0x7efddc098b50 msgr2=0x7efddc06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:42.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.109+0000 7efde08e4700 1 -- 172.21.15.12:0/714417725 shutdown_connections 2023-07-08T12:38:42.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.109+0000 7efde08e4700 1 -- 172.21.15.12:0/714417725 wait complete. 2023-07-08T12:38:42.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.111+0000 7efde08e4700 1 Processor -- start 2023-07-08T12:38:42.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.111+0000 7efde08e4700 1 -- start start 2023-07-08T12:38:42.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efde08e4700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc0cc240 0x7efddc159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efde08e4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efddc156a10 0x7efddc1f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efde08e4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc15d390 0x7efddc14bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efde08e4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0b4070 con 0x7efddc156a10 2023-07-08T12:38:42.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efde08e4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc159f30 con 0x7efddc15d390 2023-07-08T12:38:42.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efde08e4700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0b4bd0 con 0x7efddc0cc240 2023-07-08T12:38:42.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdd9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efddc156a10 0x7efddc1f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdd9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efddc156a10 0x7efddc1f65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50156/0 (socket says 172.21.15.12:50156) 2023-07-08T12:38:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdd9d9b700 1 -- 172.21.15.12:0/3274616904 learned_addr learned my addr 172.21.15.12:0/3274616904 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdda59c700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc0cc240 0x7efddc159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efddad9d700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc15d390 0x7efddc14bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdd9d9b700 1 -- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc0cc240 msgr2=0x7efddc159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdd9d9b700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc0cc240 0x7efddc159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdd9d9b700 1 -- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc15d390 msgr2=0x7efddc14bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdd9d9b700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc15d390 0x7efddc14bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdd9d9b700 1 -- 172.21.15.12:0/3274616904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd4009a70 con 0x7efddc156a10 2023-07-08T12:38:42.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdda59c700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc0cc240 0x7efddc159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:42.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdd9d9b700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efddc156a10 0x7efddc1f65f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efdd000c750 tx=0x7efdd000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:42.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdcaffd700 1 -- 172.21.15.12:0/3274616904 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0004c80 con 0x7efddc156a10 2023-07-08T12:38:42.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdcaffd700 1 -- 172.21.15.12:0/3274616904 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd0004e00 con 0x7efddc156a10 2023-07-08T12:38:42.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efde08e4700 1 -- 172.21.15.12:0/3274616904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc1f6b30 con 0x7efddc156a10 2023-07-08T12:38:42.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.112+0000 7efdcaffd700 1 -- 172.21.15.12:0/3274616904 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0007310 con 0x7efddc156a10 2023-07-08T12:38:42.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.113+0000 7efde08e4700 1 -- 172.21.15.12:0/3274616904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc0b42a0 con 0x7efddc156a10 2023-07-08T12:38:42.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.113+0000 7efde08e4700 1 -- 172.21.15.12:0/3274616904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc04ea60 con 0x7efddc156a10 2023-07-08T12:38:42.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.114+0000 7efdcaffd700 1 -- 172.21.15.12:0/3274616904 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efdd0007470 con 0x7efddc156a10 2023-07-08T12:38:42.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.114+0000 7efdcaffd700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efdbc06ace0 0x7efdbc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.114+0000 7efdcaffd700 1 -- 172.21.15.12:0/3274616904 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efdd0089480 con 0x7efddc156a10 2023-07-08T12:38:42.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.115+0000 7efdda59c700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efdbc06ace0 0x7efdbc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.115+0000 7efdda59c700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efdbc06ace0 0x7efdbc06d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009c10 tx=0x7efdcc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:42.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.119+0000 7efdcaffd700 1 -- 172.21.15.12:0/3274616904 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efdd0054240 con 0x7efddc156a10 2023-07-08T12:38:42.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.315+0000 7efde08e4700 1 -- 172.21.15.12:0/3274616904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efddc0cbef0 con 0x7efddc156a10 2023-07-08T12:38:42.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.316+0000 7efdcaffd700 1 -- 172.21.15.12:0/3274616904 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7efdd0057860 con 0x7efddc156a10 2023-07-08T12:38:42.318 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:38:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.318+0000 7efde08e4700 1 -- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efdbc06ace0 msgr2=0x7efdbc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.318+0000 7efde08e4700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efdbc06ace0 0x7efdbc06d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009c10 tx=0x7efdcc008040 comp rx=0 tx=0).stop 2023-07-08T12:38:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.318+0000 7efde08e4700 1 -- 172.21.15.12:0/3274616904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efddc156a10 msgr2=0x7efddc1f65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.318+0000 7efde08e4700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efddc156a10 0x7efddc1f65f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efdd000c750 tx=0x7efdd000ca60 comp rx=0 tx=0).stop 2023-07-08T12:38:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.318+0000 7efde08e4700 1 -- 172.21.15.12:0/3274616904 shutdown_connections 2023-07-08T12:38:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.318+0000 7efde08e4700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efdbc06ace0 0x7efdbc06d190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.318+0000 7efde08e4700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc15d390 0x7efddc14bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.318+0000 7efde08e4700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc0cc240 0x7efddc159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.318+0000 7efde08e4700 1 --2- 172.21.15.12:0/3274616904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efddc156a10 0x7efddc1f65f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.318+0000 7efde08e4700 1 -- 172.21.15.12:0/3274616904 >> 172.21.15.12:0/3274616904 conn(0x7efddc098b50 msgr2=0x7efddc15ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.318+0000 7efde08e4700 1 -- 172.21.15.12:0/3274616904 shutdown_connections 2023-07-08T12:38:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.318+0000 7efde08e4700 1 -- 172.21.15.12:0/3274616904 wait complete. 2023-07-08T12:38:42.332 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:38:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.459+0000 7f78e3438700 1 Processor -- start 2023-07-08T12:38:42.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.459+0000 7f78e3438700 1 -- start start 2023-07-08T12:38:42.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e3438700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f78dc156a10 0x7f78dc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e3438700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78dc15d390 0x7f78dc1608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e3438700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f78dc0cc240 0x7f78dc0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e3438700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78dc0ccc00 con 0x7f78dc156a10 2023-07-08T12:38:42.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e3438700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78dc160410 con 0x7f78dc0cc240 2023-07-08T12:38:42.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e3438700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc156de0 con 0x7f78dc15d390 2023-07-08T12:38:42.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e11d4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f78dc156a10 0x7f78dc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e11d4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f78dc156a10 0x7f78dc06e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50162/0 (socket says 172.21.15.12:50162) 2023-07-08T12:38:42.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e11d4700 1 -- 172.21.15.12:0/23269355 learned_addr learned my addr 172.21.15.12:0/23269355 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:42.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e19d5700 1 --2- 172.21.15.12:0/23269355 >> v2:172.21.15.160:3300/0 conn(0x7f78dc0cc240 0x7f78dc0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e09d3700 1 --2- 172.21.15.12:0/23269355 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78dc15d390 0x7f78dc1608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e11d4700 1 -- 172.21.15.12:0/23269355 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78dc15d390 msgr2=0x7f78dc1608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:42.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e11d4700 1 --2- 172.21.15.12:0/23269355 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78dc15d390 0x7f78dc1608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e11d4700 1 -- 172.21.15.12:0/23269355 >> v2:172.21.15.160:3300/0 conn(0x7f78dc0cc240 msgr2=0x7f78dc0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:42.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e11d4700 1 --2- 172.21.15.12:0/23269355 >> v2:172.21.15.160:3300/0 conn(0x7f78dc0cc240 0x7f78dc0cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e11d4700 1 -- 172.21.15.12:0/23269355 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc06ec70 con 0x7f78dc156a10 2023-07-08T12:38:42.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e19d5700 1 --2- 172.21.15.12:0/23269355 >> v2:172.21.15.160:3300/0 conn(0x7f78dc0cc240 0x7f78dc0cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:42.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.460+0000 7f78e11d4700 1 --2- 172.21.15.12:0/23269355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78dc156a10 0x7f78dc06e0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f78dc0cde00 tx=0x7f78d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d7597c970e0160 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:42.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.461+0000 7f78d3fff700 1 -- 172.21.15.12:0/23269355 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8005e30 con 0x7f78dc156a10 2023-07-08T12:38:42.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.461+0000 7f78d3fff700 1 -- 172.21.15.12:0/23269355 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d8004030 con 0x7f78dc156a10 2023-07-08T12:38:42.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.461+0000 7f78d3fff700 1 -- 172.21.15.12:0/23269355 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d80043e0 con 0x7f78dc156a10 2023-07-08T12:38:42.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.461+0000 7f78e3438700 1 -- 172.21.15.12:0/23269355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78dc156a10 msgr2=0x7f78dc06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.461+0000 7f78e3438700 1 --2- 172.21.15.12:0/23269355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78dc156a10 0x7f78dc06e0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f78dc0cde00 tx=0x7f78d8009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.461+0000 7f78e3438700 1 -- 172.21.15.12:0/23269355 shutdown_connections 2023-07-08T12:38:42.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.461+0000 7f78e3438700 1 --2- 172.21.15.12:0/23269355 >> v2:172.21.15.160:3300/0 conn(0x7f78dc0cc240 0x7f78dc0cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.461+0000 7f78e3438700 1 --2- 172.21.15.12:0/23269355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78dc156a10 0x7f78dc06e0c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.461+0000 7f78e3438700 1 --2- 172.21.15.12:0/23269355 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78dc15d390 0x7f78dc1608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.461+0000 7f78e3438700 1 -- 172.21.15.12:0/23269355 >> 172.21.15.12:0/23269355 conn(0x7f78dc098b50 msgr2=0x7f78dc06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.461+0000 7f78e3438700 1 -- 172.21.15.12:0/23269355 shutdown_connections 2023-07-08T12:38:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.461+0000 7f78e3438700 1 -- 172.21.15.12:0/23269355 wait complete. 2023-07-08T12:38:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.463+0000 7f78e3438700 1 Processor -- start 2023-07-08T12:38:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.463+0000 7f78e3438700 1 -- start start 2023-07-08T12:38:42.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.463+0000 7f78e3438700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78dc0cc240 0x7f78dc0b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.463+0000 7f78e3438700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78dc156a10 0x7f78dc1f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.463+0000 7f78e3438700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78dc15d390 0x7f78dc15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.463+0000 7f78e3438700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc15d910 con 0x7f78dc0cc240 2023-07-08T12:38:42.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.463+0000 7f78e3438700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc15dbf0 con 0x7f78dc156a10 2023-07-08T12:38:42.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.463+0000 7f78e3438700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc15ded0 con 0x7f78dc15d390 2023-07-08T12:38:42.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.463+0000 7f78e11d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78dc0cc240 0x7f78dc0b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.463+0000 7f78e11d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78dc0cc240 0x7f78dc0b28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50168/0 (socket says 172.21.15.12:50168) 2023-07-08T12:38:42.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.463+0000 7f78e11d4700 1 -- 172.21.15.12:0/3611203593 learned_addr learned my addr 172.21.15.12:0/3611203593 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:42.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78e09d3700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78dc156a10 0x7f78dc1f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78e19d5700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78dc15d390 0x7f78dc15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78e11d4700 1 -- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78dc15d390 msgr2=0x7f78dc15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78e11d4700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78dc15d390 0x7f78dc15e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78e11d4700 1 -- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78dc156a10 msgr2=0x7f78dc1f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78e11d4700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78dc156a10 0x7f78dc1f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78e11d4700 1 -- 172.21.15.12:0/3611203593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d8009500 con 0x7f78dc0cc240 2023-07-08T12:38:42.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78e09d3700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78dc156a10 0x7f78dc1f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:42.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78e11d4700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78dc0cc240 0x7f78dc0b28d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009130 tx=0x7f78d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:42.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78d1ffb700 1 -- 172.21.15.12:0/3611203593 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8003e50 con 0x7f78dc0cc240 2023-07-08T12:38:42.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78d1ffb700 1 -- 172.21.15.12:0/3611203593 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d801d430 con 0x7f78dc0cc240 2023-07-08T12:38:42.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78e3438700 1 -- 172.21.15.12:0/3611203593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78dc15e1b0 con 0x7f78dc0cc240 2023-07-08T12:38:42.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78d1ffb700 1 -- 172.21.15.12:0/3611203593 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d801d7c0 con 0x7f78dc0cc240 2023-07-08T12:38:42.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.464+0000 7f78e3438700 1 -- 172.21.15.12:0/3611203593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78dc0b2e10 con 0x7f78dc0cc240 2023-07-08T12:38:42.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.465+0000 7f78e3438700 1 -- 172.21.15.12:0/3611203593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78dc15e6d0 con 0x7f78dc0cc240 2023-07-08T12:38:42.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.466+0000 7f78d1ffb700 1 -- 172.21.15.12:0/3611203593 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f78d801d920 con 0x7f78dc0cc240 2023-07-08T12:38:42.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.466+0000 7f78d1ffb700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f78c006ace0 0x7f78c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.466+0000 7f78d1ffb700 1 -- 172.21.15.12:0/3611203593 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f78d808ab90 con 0x7f78dc0cc240 2023-07-08T12:38:42.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.467+0000 7f78e09d3700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f78c006ace0 0x7f78c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.467+0000 7f78e09d3700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f78c006ace0 0x7f78c006d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0099f0 tx=0x7f78cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:42.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.471+0000 7f78d1ffb700 1 -- 172.21.15.12:0/3611203593 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f78dc15e6d0 con 0x7f78dc0cc240 2023-07-08T12:38:42.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.670+0000 7f78e3438700 1 -- 172.21.15.12:0/3611203593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f78dc15e6d0 con 0x7f78dc0cc240 2023-07-08T12:38:42.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.670+0000 7f78d1ffb700 1 -- 172.21.15.12:0/3611203593 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f78dc15e6d0 con 0x7f78dc0cc240 2023-07-08T12:38:42.674 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:38:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.673+0000 7f78e3438700 1 -- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f78c006ace0 msgr2=0x7f78c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.673+0000 7f78e3438700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f78c006ace0 0x7f78c006d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0099f0 tx=0x7f78cc008040 comp rx=0 tx=0).stop 2023-07-08T12:38:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.673+0000 7f78e3438700 1 -- 172.21.15.12:0/3611203593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78dc0cc240 msgr2=0x7f78dc0b28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.673+0000 7f78e3438700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78dc0cc240 0x7f78dc0b28d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009130 tx=0x7f78d8004750 comp rx=0 tx=0).stop 2023-07-08T12:38:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.673+0000 7f78e3438700 1 -- 172.21.15.12:0/3611203593 shutdown_connections 2023-07-08T12:38:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.673+0000 7f78e3438700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f78c006ace0 0x7f78c006d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.673+0000 7f78e3438700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78dc15d390 0x7f78dc15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.673+0000 7f78e3438700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78dc0cc240 0x7f78dc0b28d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.673+0000 7f78e3438700 1 --2- 172.21.15.12:0/3611203593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78dc156a10 0x7f78dc1f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.673+0000 7f78e3438700 1 -- 172.21.15.12:0/3611203593 >> 172.21.15.12:0/3611203593 conn(0x7f78dc098b50 msgr2=0x7f78dc0ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.673+0000 7f78e3438700 1 -- 172.21.15.12:0/3611203593 shutdown_connections 2023-07-08T12:38:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.673+0000 7f78e3438700 1 -- 172.21.15.12:0/3611203593 wait complete. 2023-07-08T12:38:42.687 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-08T12:38:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.813+0000 7f361d86d700 1 Processor -- start 2023-07-08T12:38:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.813+0000 7f361d86d700 1 -- start start 2023-07-08T12:38:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f361d86d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f36180d3730 0x7f36180cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f361d86d700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f36180cfa90 0x7f36180d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f361d86d700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36180d0b20 con 0x7f36180b8430 2023-07-08T12:38:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f361d86d700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36180cda30 con 0x7f36180d3730 2023-07-08T12:38:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f361d86d700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36180b8800 con 0x7f36180cfa90 2023-07-08T12:38:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f36167fc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f36180d3730 0x7f36180cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f36167fc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f36180d3730 0x7f36180cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50178/0 (socket says 172.21.15.12:50178) 2023-07-08T12:38:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f36167fc700 1 -- 172.21.15.12:0/2893098322 learned_addr learned my addr 172.21.15.12:0/2893098322 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f36177fe700 1 --2- 172.21.15.12:0/2893098322 >> v2:172.21.15.160:3300/0 conn(0x7f36180cfa90 0x7f36180d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f36167fc700 1 -- 172.21.15.12:0/2893098322 >> v2:172.21.15.160:3300/0 conn(0x7f36180cfa90 msgr2=0x7f36180d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f36167fc700 1 --2- 172.21.15.12:0/2893098322 >> v2:172.21.15.160:3300/0 conn(0x7f36180cfa90 0x7f36180d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f36167fc700 1 -- 172.21.15.12:0/2893098322 >> v1:172.21.15.160:6789/0 conn(0x7f36180b8430 legacy=0x7f36180cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f36167fc700 1 -- 172.21.15.12:0/2893098322 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36180cfe90 con 0x7f36180d3730 2023-07-08T12:38:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f36177fe700 1 --2- 172.21.15.12:0/2893098322 >> v2:172.21.15.160:3300/0 conn(0x7f36180cfa90 0x7f36180d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.814+0000 7f36167fc700 1 --2- 172.21.15.12:0/2893098322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36180d3730 0x7f36180cf550 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f36180b2100 tx=0x7f3608009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c59ecea563b2c14 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.815+0000 7f3615ffb700 1 -- 172.21.15.12:0/2893098322 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800b870 con 0x7f36180d3730 2023-07-08T12:38:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.815+0000 7f3615ffb700 1 -- 172.21.15.12:0/2893098322 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360800b9d0 con 0x7f36180d3730 2023-07-08T12:38:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.815+0000 7f3615ffb700 1 -- 172.21.15.12:0/2893098322 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800bd60 con 0x7f36180d3730 2023-07-08T12:38:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.815+0000 7f361d86d700 1 -- 172.21.15.12:0/2893098322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36180d3730 msgr2=0x7f36180cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.815+0000 7f361d86d700 1 --2- 172.21.15.12:0/2893098322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36180d3730 0x7f36180cf550 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f36180b2100 tx=0x7f3608009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.815+0000 7f361d86d700 1 -- 172.21.15.12:0/2893098322 shutdown_connections 2023-07-08T12:38:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.815+0000 7f361d86d700 1 --2- 172.21.15.12:0/2893098322 >> v2:172.21.15.160:3300/0 conn(0x7f36180cfa90 0x7f36180d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.815+0000 7f361d86d700 1 --2- 172.21.15.12:0/2893098322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36180d3730 0x7f36180cf550 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.815+0000 7f361d86d700 1 -- 172.21.15.12:0/2893098322 >> 172.21.15.12:0/2893098322 conn(0x7f361806b3f0 msgr2=0x7f361806b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.815+0000 7f361d86d700 1 -- 172.21.15.12:0/2893098322 shutdown_connections 2023-07-08T12:38:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.815+0000 7f361d86d700 1 -- 172.21.15.12:0/2893098322 wait complete. 2023-07-08T12:38:42.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.816+0000 7f361d86d700 1 Processor -- start 2023-07-08T12:38:42.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.817+0000 7f361d86d700 1 -- start start 2023-07-08T12:38:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.817+0000 7f361d86d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36180b8430 0x7f36180cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.817+0000 7f361d86d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36180cfa90 0x7f36181f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.817+0000 7f361d86d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36180d3730 0x7f3618148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.817+0000 7f361d86d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36180cbee0 con 0x7f36180b8430 2023-07-08T12:38:42.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.817+0000 7f361d86d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36181f2f20 con 0x7f36180d3730 2023-07-08T12:38:42.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.817+0000 7f361d86d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36180cc160 con 0x7f36180cfa90 2023-07-08T12:38:42.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.817+0000 7f3616ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36180b8430 0x7f36180cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.817+0000 7f3616ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36180b8430 0x7f36180cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50182/0 (socket says 172.21.15.12:50182) 2023-07-08T12:38:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.817+0000 7f3616ffd700 1 -- 172.21.15.12:0/2535462576 learned_addr learned my addr 172.21.15.12:0/2535462576 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.818+0000 7f36177fe700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36180d3730 0x7f3618148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.818+0000 7f36167fc700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36180cfa90 0x7f36181f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.818+0000 7f3616ffd700 1 -- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36180cfa90 msgr2=0x7f36181f29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.818+0000 7f3616ffd700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36180cfa90 0x7f36181f29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.818+0000 7f3616ffd700 1 -- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36180d3730 msgr2=0x7f3618148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:42.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.818+0000 7f3616ffd700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36180d3730 0x7f3618148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:42.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.818+0000 7f3616ffd700 1 -- 172.21.15.12:0/2535462576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3608009550 con 0x7f36180b8430 2023-07-08T12:38:42.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.818+0000 7f3616ffd700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36180b8430 0x7f36180cd200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f360c00c510 tx=0x7f360c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:42.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.818+0000 7f35fffff700 1 -- 172.21.15.12:0/2535462576 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c0052d0 con 0x7f36180b8430 2023-07-08T12:38:42.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.818+0000 7f35fffff700 1 -- 172.21.15.12:0/2535462576 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c005430 con 0x7f36180b8430 2023-07-08T12:38:42.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.818+0000 7f361d86d700 1 -- 172.21.15.12:0/2535462576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36180cc440 con 0x7f36180b8430 2023-07-08T12:38:42.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.818+0000 7f35fffff700 1 -- 172.21.15.12:0/2535462576 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c0038c0 con 0x7f36180b8430 2023-07-08T12:38:42.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.818+0000 7f361d86d700 1 -- 172.21.15.12:0/2535462576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36180cca00 con 0x7f36180b8430 2023-07-08T12:38:42.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.819+0000 7f361d86d700 1 -- 172.21.15.12:0/2535462576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361804ea60 con 0x7f36180b8430 2023-07-08T12:38:42.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.819+0000 7f35fffff700 1 -- 172.21.15.12:0/2535462576 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f360c003a20 con 0x7f36180b8430 2023-07-08T12:38:42.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.820+0000 7f35fffff700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f360406ace0 0x7f360406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:42.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.820+0000 7f35fffff700 1 -- 172.21.15.12:0/2535462576 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f360c088c00 con 0x7f36180b8430 2023-07-08T12:38:42.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.820+0000 7f36167fc700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f360406ace0 0x7f360406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:42.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.821+0000 7f36167fc700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f360406ace0 0x7f360406d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3608014040 tx=0x7f3608008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:42.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:42.824+0000 7f35fffff700 1 -- 172.21.15.12:0/2535462576 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f360c08f080 con 0x7f36180b8430 2023-07-08T12:38:43.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.020+0000 7f361d86d700 1 -- 172.21.15.12:0/2535462576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36180b1710 con 0x7f36180b8430 2023-07-08T12:38:43.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.021+0000 7f35fffff700 1 -- 172.21.15.12:0/2535462576 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f360c053a50 con 0x7f36180b8430 2023-07-08T12:38:43.024 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:38:43.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.023+0000 7f361d86d700 1 -- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f360406ace0 msgr2=0x7f360406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.023+0000 7f361d86d700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f360406ace0 0x7f360406d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3608014040 tx=0x7f3608008030 comp rx=0 tx=0).stop 2023-07-08T12:38:43.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.023+0000 7f361d86d700 1 -- 172.21.15.12:0/2535462576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36180b8430 msgr2=0x7f36180cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.023+0000 7f361d86d700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36180b8430 0x7f36180cd200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f360c00c510 tx=0x7f360c00c8d0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.023+0000 7f361d86d700 1 -- 172.21.15.12:0/2535462576 shutdown_connections 2023-07-08T12:38:43.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.023+0000 7f361d86d700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f360406ace0 0x7f360406d190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.023+0000 7f361d86d700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36180d3730 0x7f3618148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.023+0000 7f361d86d700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36180b8430 0x7f36180cd200 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.023+0000 7f361d86d700 1 --2- 172.21.15.12:0/2535462576 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36180cfa90 0x7f36181f29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.023+0000 7f361d86d700 1 -- 172.21.15.12:0/2535462576 >> 172.21.15.12:0/2535462576 conn(0x7f361806b3f0 msgr2=0x7f361815fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:43.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.023+0000 7f361d86d700 1 -- 172.21.15.12:0/2535462576 shutdown_connections 2023-07-08T12:38:43.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.023+0000 7f361d86d700 1 -- 172.21.15.12:0/2535462576 wait complete. 2023-07-08T12:38:43.037 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-08T12:38:43.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.162+0000 7fdeedb5a700 1 Processor -- start 2023-07-08T12:38:43.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.162+0000 7fdeedb5a700 1 -- start start 2023-07-08T12:38:43.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.163+0000 7fdeedb5a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdee806ebe0 0x7fdee80cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.163+0000 7fdeedb5a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdee80cd9b0 0x7fdee80cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.163+0000 7fdeedb5a700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee815d770 con 0x7fdee80aeec0 2023-07-08T12:38:43.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.163+0000 7fdeedb5a700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee806d700 con 0x7fdee806ebe0 2023-07-08T12:38:43.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.163+0000 7fdeedb5a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee815a600 con 0x7fdee80cd9b0 2023-07-08T12:38:43.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.163+0000 7fdee6ffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdee806ebe0 0x7fdee80cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.163+0000 7fdee6ffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdee806ebe0 0x7fdee80cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50192/0 (socket says 172.21.15.12:50192) 2023-07-08T12:38:43.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.163+0000 7fdee6ffd700 1 -- 172.21.15.12:0/3041015882 learned_addr learned my addr 172.21.15.12:0/3041015882 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:43.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.164+0000 7fdee7fff700 1 --2- 172.21.15.12:0/3041015882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdee80cd9b0 0x7fdee80cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.164+0000 7fdee6ffd700 1 -- 172.21.15.12:0/3041015882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdee80cd9b0 msgr2=0x7fdee80cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.164+0000 7fdee6ffd700 1 --2- 172.21.15.12:0/3041015882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdee80cd9b0 0x7fdee80cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.164+0000 7fdee6ffd700 1 -- 172.21.15.12:0/3041015882 >> v1:172.21.15.160:6789/0 conn(0x7fdee80aeec0 legacy=0x7fdee806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.164+0000 7fdee6ffd700 1 -- 172.21.15.12:0/3041015882 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80cddc0 con 0x7fdee806ebe0 2023-07-08T12:38:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.164+0000 7fdee7fff700 1 --2- 172.21.15.12:0/3041015882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdee80cd9b0 0x7fdee80cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.164+0000 7fdee6ffd700 1 --2- 172.21.15.12:0/3041015882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee806ebe0 0x7fdee80cd470 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fded80098f0 tx=0x7fded8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173304544ff6d760 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.164+0000 7fdee67fc700 1 -- 172.21.15.12:0/3041015882 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded800b8a0 con 0x7fdee806ebe0 2023-07-08T12:38:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.164+0000 7fdee67fc700 1 -- 172.21.15.12:0/3041015882 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded800ba00 con 0x7fdee806ebe0 2023-07-08T12:38:43.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.164+0000 7fdee67fc700 1 -- 172.21.15.12:0/3041015882 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded800bd90 con 0x7fdee806ebe0 2023-07-08T12:38:43.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.165+0000 7fdeedb5a700 1 -- 172.21.15.12:0/3041015882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee806ebe0 msgr2=0x7fdee80cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.165+0000 7fdeedb5a700 1 --2- 172.21.15.12:0/3041015882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee806ebe0 0x7fdee80cd470 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fded80098f0 tx=0x7fded8009c00 comp rx=0 tx=0).stop 2023-07-08T12:38:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.165+0000 7fdeedb5a700 1 -- 172.21.15.12:0/3041015882 shutdown_connections 2023-07-08T12:38:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.165+0000 7fdeedb5a700 1 --2- 172.21.15.12:0/3041015882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdee80cd9b0 0x7fdee80cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.165+0000 7fdeedb5a700 1 --2- 172.21.15.12:0/3041015882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee806ebe0 0x7fdee80cd470 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.165+0000 7fdeedb5a700 1 -- 172.21.15.12:0/3041015882 >> 172.21.15.12:0/3041015882 conn(0x7fdee8098b50 msgr2=0x7fdee806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:43.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.165+0000 7fdeedb5a700 1 -- 172.21.15.12:0/3041015882 shutdown_connections 2023-07-08T12:38:43.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.165+0000 7fdeedb5a700 1 -- 172.21.15.12:0/3041015882 wait complete. 2023-07-08T12:38:43.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.166+0000 7fdeedb5a700 1 Processor -- start 2023-07-08T12:38:43.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.166+0000 7fdeedb5a700 1 -- start start 2023-07-08T12:38:43.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdeedb5a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdee806ebe0 0x7fdee80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdeedb5a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdee80aeec0 0x7fdee80c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdeedb5a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80cd9b0 0x7fdee80c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdeedb5a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80cb780 con 0x7fdee80cd9b0 2023-07-08T12:38:43.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdeedb5a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80c7b30 con 0x7fdee806ebe0 2023-07-08T12:38:43.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdeedb5a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee814b1d0 con 0x7fdee80aeec0 2023-07-08T12:38:43.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdee7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80cd9b0 0x7fdee80c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdee7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80cd9b0 0x7fdee80c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50200/0 (socket says 172.21.15.12:50200) 2023-07-08T12:38:43.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdee7fff700 1 -- 172.21.15.12:0/2540618669 learned_addr learned my addr 172.21.15.12:0/2540618669 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:43.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdee6ffd700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdee80aeec0 0x7fdee80c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdee77fe700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdee806ebe0 0x7fdee80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdee7fff700 1 -- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdee80aeec0 msgr2=0x7fdee80c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdee7fff700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdee80aeec0 0x7fdee80c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdee7fff700 1 -- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdee806ebe0 msgr2=0x7fdee80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdee7fff700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdee806ebe0 0x7fdee80b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdee7fff700 1 -- 172.21.15.12:0/2540618669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded8009550 con 0x7fdee80cd9b0 2023-07-08T12:38:43.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.167+0000 7fdee77fe700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdee806ebe0 0x7fdee80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:43.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.168+0000 7fdee7fff700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80cd9b0 0x7fdee80c8500 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdee0007ea0 tx=0x7fdee000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:43.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.168+0000 7fdecffff700 1 -- 172.21.15.12:0/2540618669 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee000ddd0 con 0x7fdee80cd9b0 2023-07-08T12:38:43.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.168+0000 7fdecffff700 1 -- 172.21.15.12:0/2540618669 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee000f040 con 0x7fdee80cd9b0 2023-07-08T12:38:43.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.168+0000 7fdecffff700 1 -- 172.21.15.12:0/2540618669 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0003c60 con 0x7fdee80cd9b0 2023-07-08T12:38:43.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.168+0000 7fdeedb5a700 1 -- 172.21.15.12:0/2540618669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee80b2100 con 0x7fdee80cd9b0 2023-07-08T12:38:43.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.168+0000 7fdeedb5a700 1 -- 172.21.15.12:0/2540618669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee80c74e0 con 0x7fdee80cd9b0 2023-07-08T12:38:43.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.169+0000 7fdeedb5a700 1 -- 172.21.15.12:0/2540618669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee80c7340 con 0x7fdee80cd9b0 2023-07-08T12:38:43.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.169+0000 7fdecffff700 1 -- 172.21.15.12:0/2540618669 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdee0004030 con 0x7fdee80cd9b0 2023-07-08T12:38:43.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.170+0000 7fdecffff700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fded406ace0 0x7fded406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.170+0000 7fdecffff700 1 -- 172.21.15.12:0/2540618669 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdee0089920 con 0x7fdee80cd9b0 2023-07-08T12:38:43.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.170+0000 7fdee77fe700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fded406ace0 0x7fded406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.170+0000 7fdee77fe700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fded406ace0 0x7fded406d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdedc000fb0 tx=0x7fdedc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:43.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.174+0000 7fdecffff700 1 -- 172.21.15.12:0/2540618669 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdee80c7340 con 0x7fdee80cd9b0 2023-07-08T12:38:43.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.373+0000 7fdeedb5a700 1 -- 172.21.15.12:0/2540618669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdee80c7340 con 0x7fdee80cd9b0 2023-07-08T12:38:43.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.373+0000 7fdecffff700 1 -- 172.21.15.12:0/2540618669 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdee80c7340 con 0x7fdee80cd9b0 2023-07-08T12:38:43.376 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-07-08T12:38:43.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.375+0000 7fdeedb5a700 1 -- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fded406ace0 msgr2=0x7fded406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.375+0000 7fdeedb5a700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fded406ace0 0x7fded406d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdedc000fb0 tx=0x7fdedc009040 comp rx=0 tx=0).stop 2023-07-08T12:38:43.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.375+0000 7fdeedb5a700 1 -- 172.21.15.12:0/2540618669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80cd9b0 msgr2=0x7fdee80c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.375+0000 7fdeedb5a700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80cd9b0 0x7fdee80c8500 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdee0007ea0 tx=0x7fdee000d3b0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.375+0000 7fdeedb5a700 1 -- 172.21.15.12:0/2540618669 shutdown_connections 2023-07-08T12:38:43.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.375+0000 7fdeedb5a700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fded406ace0 0x7fded406d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.375+0000 7fdeedb5a700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdee80cd9b0 0x7fdee80c8500 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.375+0000 7fdeedb5a700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdee806ebe0 0x7fdee80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.375+0000 7fdeedb5a700 1 --2- 172.21.15.12:0/2540618669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdee80aeec0 0x7fdee80c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.375+0000 7fdeedb5a700 1 -- 172.21.15.12:0/2540618669 >> 172.21.15.12:0/2540618669 conn(0x7fdee8098b50 msgr2=0x7fdee80cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:43.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.375+0000 7fdeedb5a700 1 -- 172.21.15.12:0/2540618669 shutdown_connections 2023-07-08T12:38:43.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.375+0000 7fdeedb5a700 1 -- 172.21.15.12:0/2540618669 wait complete. 2023-07-08T12:38:43.391 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-08T12:38:43.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.517+0000 7f8359374700 1 Processor -- start 2023-07-08T12:38:43.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.517+0000 7f8359374700 1 -- start start 2023-07-08T12:38:43.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.518+0000 7f8359374700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8354160a40 0x7f835406bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.518+0000 7f8359374700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f835406c300 0x7f83540caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.518+0000 7f8359374700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83540b0070 con 0x7f83540afca0 2023-07-08T12:38:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.518+0000 7f8359374700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8354160eb0 con 0x7f835406c300 2023-07-08T12:38:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.518+0000 7f8359374700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835406e880 con 0x7f8354160a40 2023-07-08T12:38:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.518+0000 7f83537fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f835406c300 0x7f83540caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.518+0000 7f83537fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f835406c300 0x7f83540caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50208/0 (socket says 172.21.15.12:50208) 2023-07-08T12:38:43.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.518+0000 7f83537fe700 1 -- 172.21.15.12:0/1872597068 learned_addr learned my addr 172.21.15.12:0/1872597068 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:43.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.518+0000 7f83527fc700 1 --2- 172.21.15.12:0/1872597068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8354160a40 0x7f835406bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.518+0000 7f83537fe700 1 -- 172.21.15.12:0/1872597068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8354160a40 msgr2=0x7f835406bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:43.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.518+0000 7f83537fe700 1 --2- 172.21.15.12:0/1872597068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8354160a40 0x7f835406bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.518+0000 7f83537fe700 1 -- 172.21.15.12:0/1872597068 >> v1:172.21.15.160:6789/0 conn(0x7f83540afca0 legacy=0x7f835406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.518+0000 7f83537fe700 1 -- 172.21.15.12:0/1872597068 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835406eb60 con 0x7f835406c300 2023-07-08T12:38:43.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.519+0000 7f83537fe700 1 --2- 172.21.15.12:0/1872597068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f835406c300 0x7f83540caee0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f834c009e50 tx=0x7f834c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f975cd1064008a4a server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:43.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.519+0000 7f8351ffb700 1 -- 172.21.15.12:0/1872597068 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c00e070 con 0x7f835406c300 2023-07-08T12:38:43.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.519+0000 7f8351ffb700 1 -- 172.21.15.12:0/1872597068 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c00b910 con 0x7f835406c300 2023-07-08T12:38:43.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.519+0000 7f8351ffb700 1 -- 172.21.15.12:0/1872597068 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c00bcc0 con 0x7f835406c300 2023-07-08T12:38:43.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.519+0000 7f8359374700 1 -- 172.21.15.12:0/1872597068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f835406c300 msgr2=0x7f83540caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.519+0000 7f8359374700 1 --2- 172.21.15.12:0/1872597068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f835406c300 0x7f83540caee0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f834c009e50 tx=0x7f834c00b330 comp rx=0 tx=0).stop 2023-07-08T12:38:43.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.519+0000 7f8359374700 1 -- 172.21.15.12:0/1872597068 shutdown_connections 2023-07-08T12:38:43.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.519+0000 7f8359374700 1 --2- 172.21.15.12:0/1872597068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f835406c300 0x7f83540caee0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.519+0000 7f8359374700 1 --2- 172.21.15.12:0/1872597068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8354160a40 0x7f835406bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.519+0000 7f8359374700 1 -- 172.21.15.12:0/1872597068 >> 172.21.15.12:0/1872597068 conn(0x7f835408a3b0 msgr2=0x7f835406a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:43.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.519+0000 7f8359374700 1 -- 172.21.15.12:0/1872597068 shutdown_connections 2023-07-08T12:38:43.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.519+0000 7f8359374700 1 -- 172.21.15.12:0/1872597068 wait complete. 2023-07-08T12:38:43.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.521+0000 7f8359374700 1 Processor -- start 2023-07-08T12:38:43.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.521+0000 7f8359374700 1 -- start start 2023-07-08T12:38:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.522+0000 7f8359374700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f835406c300 0x7f83540c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.522+0000 7f8359374700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83540afca0 0x7f83540b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.522+0000 7f8359374700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354160a40 0x7f83540b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.522+0000 7f8359374700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83540c8230 con 0x7f8354160a40 2023-07-08T12:38:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.522+0000 7f8359374700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83541e7340 con 0x7f83540afca0 2023-07-08T12:38:43.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.522+0000 7f8359374700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83541e75e0 con 0x7f835406c300 2023-07-08T12:38:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.522+0000 7f83537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354160a40 0x7f83540b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.522+0000 7f83537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354160a40 0x7f83540b5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50214/0 (socket says 172.21.15.12:50214) 2023-07-08T12:38:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.522+0000 7f83537fe700 1 -- 172.21.15.12:0/2998754553 learned_addr learned my addr 172.21.15.12:0/2998754553 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.522+0000 7f8352ffd700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f835406c300 0x7f83540c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.522+0000 7f83527fc700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83540afca0 0x7f83540b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.523+0000 7f83537fe700 1 -- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f835406c300 msgr2=0x7f83540c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.523+0000 7f83537fe700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f835406c300 0x7f83540c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.523+0000 7f83537fe700 1 -- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83540afca0 msgr2=0x7f83540b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.523+0000 7f83537fe700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83540afca0 0x7f83540b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.523+0000 7f83537fe700 1 -- 172.21.15.12:0/2998754553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c009b30 con 0x7f8354160a40 2023-07-08T12:38:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.523+0000 7f83537fe700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354160a40 0x7f83540b5c80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f834c012000 tx=0x7f834c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.523+0000 7f833b7fe700 1 -- 172.21.15.12:0/2998754553 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c004800 con 0x7f8354160a40 2023-07-08T12:38:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.523+0000 7f833b7fe700 1 -- 172.21.15.12:0/2998754553 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c004960 con 0x7f8354160a40 2023-07-08T12:38:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.523+0000 7f8359374700 1 -- 172.21.15.12:0/2998754553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83541e78c0 con 0x7f8354160a40 2023-07-08T12:38:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.523+0000 7f833b7fe700 1 -- 172.21.15.12:0/2998754553 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c01c680 con 0x7f8354160a40 2023-07-08T12:38:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.523+0000 7f8359374700 1 -- 172.21.15.12:0/2998754553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83541e5d00 con 0x7f8354160a40 2023-07-08T12:38:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.524+0000 7f8359374700 1 -- 172.21.15.12:0/2998754553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835404ea60 con 0x7f8354160a40 2023-07-08T12:38:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.524+0000 7f833b7fe700 1 -- 172.21.15.12:0/2998754553 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f834c01c7e0 con 0x7f8354160a40 2023-07-08T12:38:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.525+0000 7f833b7fe700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f834006ace0 0x7f834006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.525+0000 7f833b7fe700 1 -- 172.21.15.12:0/2998754553 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f834c08b290 con 0x7f8354160a40 2023-07-08T12:38:43.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.525+0000 7f8352ffd700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f834006ace0 0x7f834006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.526+0000 7f8352ffd700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f834006ace0 0x7f834006d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f83541548a0 tx=0x7f8348009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:43.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.530+0000 7f833b7fe700 1 -- 172.21.15.12:0/2998754553 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f834c0560e0 con 0x7f8354160a40 2023-07-08T12:38:43.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.726+0000 7f8359374700 1 -- 172.21.15.12:0/2998754553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f835406a9d0 con 0x7f8354160a40 2023-07-08T12:38:43.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.726+0000 7f833b7fe700 1 -- 172.21.15.12:0/2998754553 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f834c059700 con 0x7f8354160a40 2023-07-08T12:38:43.729 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:38:43.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.728+0000 7f8359374700 1 -- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f834006ace0 msgr2=0x7f834006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.728+0000 7f8359374700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f834006ace0 0x7f834006d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f83541548a0 tx=0x7f8348009040 comp rx=0 tx=0).stop 2023-07-08T12:38:43.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.728+0000 7f8359374700 1 -- 172.21.15.12:0/2998754553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354160a40 msgr2=0x7f83540b5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.728+0000 7f8359374700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354160a40 0x7f83540b5c80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f834c012000 tx=0x7f834c004ad0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.728+0000 7f8359374700 1 -- 172.21.15.12:0/2998754553 shutdown_connections 2023-07-08T12:38:43.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.728+0000 7f8359374700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f834006ace0 0x7f834006d190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.728+0000 7f8359374700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8354160a40 0x7f83540b5c80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.728+0000 7f8359374700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f835406c300 0x7f83540c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.728+0000 7f8359374700 1 --2- 172.21.15.12:0/2998754553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83540afca0 0x7f83540b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.728+0000 7f8359374700 1 -- 172.21.15.12:0/2998754553 >> 172.21.15.12:0/2998754553 conn(0x7f835408a3b0 msgr2=0x7f83540cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:43.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.728+0000 7f8359374700 1 -- 172.21.15.12:0/2998754553 shutdown_connections 2023-07-08T12:38:43.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.728+0000 7f8359374700 1 -- 172.21.15.12:0/2998754553 wait complete. 2023-07-08T12:38:43.743 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-08T12:38:43.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.865+0000 7f50d96c1700 1 Processor -- start 2023-07-08T12:38:43.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.865+0000 7f50d96c1700 1 -- start start 2023-07-08T12:38:43.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.866+0000 7f50d96c1700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f50d415c460 0x7f50d40b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.866+0000 7f50d96c1700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d40d1790 con 0x7f50d40cf8f0 2023-07-08T12:38:43.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.866+0000 7f50d96c1700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d40cd950 con 0x7f50d40d2640 2023-07-08T12:38:43.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.866+0000 7f50d96c1700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d40cedd0 con 0x7f50d415c460 2023-07-08T12:38:43.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.866+0000 7f50d37fe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f50d40cf8f0 0x7f50d415fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37618/0 (socket says 172.21.15.12:37618) 2023-07-08T12:38:43.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.866+0000 7f50d37fe700 1 -- 172.21.15.12:0/1373899365 learned_addr learned my addr 172.21.15.12:0/1373899365 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:43.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.866+0000 7f50d2ffd700 1 --2- 172.21.15.12:0/1373899365 >> v2:172.21.15.160:3300/0 conn(0x7f50d415c460 0x7f50d40b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.866+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253366387 0 0) 0x7f50d40d1790 con 0x7f50d40cf8f0 2023-07-08T12:38:43.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.866+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50bc0035b0 con 0x7f50d40cf8f0 2023-07-08T12:38:43.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2781375134 0 0) 0x7f50bc0035b0 con 0x7f50d40cf8f0 2023-07-08T12:38:43.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 >> v2:172.21.15.160:3300/0 conn(0x7f50d415c460 msgr2=0x7f50d40b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:43.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d1ffb700 1 --2- 172.21.15.12:0/1373899365 >> v2:172.21.15.160:3300/0 conn(0x7f50d415c460 0x7f50d40b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 >> v1:172.21.15.160:6789/0 conn(0x7f50d40d2640 legacy=0x7f50d40cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d40d14b0 con 0x7f50d40cf8f0 2023-07-08T12:38:43.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f50cc0019f0 con 0x7f50d40cf8f0 2023-07-08T12:38:43.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 >> v1:172.21.15.12:6789/0 conn(0x7f50d40cf8f0 legacy=0x7f50d415fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d1ffb700 1 --2- 172.21.15.12:0/1373899365 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50bc003a00 0x7f50bc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d1ffb700 1 --2- 172.21.15.12:0/1373899365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc004850 0x7f50bc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d1ffb700 1 --2- 172.21.15.12:0/1373899365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50bc005320 0x7f50bc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40cedd0 con 0x7f50bc005320 2023-07-08T12:38:43.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc00c250 con 0x7f50bc004850 2023-07-08T12:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc00c410 con 0x7f50bc003a00 2023-07-08T12:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d37fe700 1 --2- 172.21.15.12:0/1373899365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50bc005320 0x7f50bc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d2ffd700 1 --2- 172.21.15.12:0/1373899365 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50bc003a00 0x7f50bc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d27fc700 1 --2- 172.21.15.12:0/1373899365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc004850 0x7f50bc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d37fe700 1 -- 172.21.15.12:0/1373899365 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50bc003a00 msgr2=0x7f50bc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d37fe700 1 --2- 172.21.15.12:0/1373899365 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50bc003a00 0x7f50bc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d37fe700 1 -- 172.21.15.12:0/1373899365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc004850 msgr2=0x7f50bc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:43.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d37fe700 1 --2- 172.21.15.12:0/1373899365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc004850 0x7f50bc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d37fe700 1 -- 172.21.15.12:0/1373899365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc0035b0 con 0x7f50bc005320 2023-07-08T12:38:43.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d37fe700 1 -- 172.21.15.12:0/1373899365 reap_dead start 2023-07-08T12:38:43.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.867+0000 7f50d37fe700 1 --2- 172.21.15.12:0/1373899365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50bc005320 0x7f50bc00ba10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00a830 tx=0x7f50bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5e1b66313bda74 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:43.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.868+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00a2f0 con 0x7f50bc005320 2023-07-08T12:38:43.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.868+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc012600 con 0x7f50bc005320 2023-07-08T12:38:43.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.868+0000 7f50d1ffb700 1 -- 172.21.15.12:0/1373899365 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc0129b0 con 0x7f50bc005320 2023-07-08T12:38:43.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.868+0000 7f50d96c1700 1 -- 172.21.15.12:0/1373899365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50bc005320 msgr2=0x7f50bc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.868+0000 7f50d96c1700 1 --2- 172.21.15.12:0/1373899365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50bc005320 0x7f50bc00ba10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00a830 tx=0x7f50bc005880 comp rx=0 tx=0).stop 2023-07-08T12:38:43.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.868+0000 7f50d96c1700 1 -- 172.21.15.12:0/1373899365 shutdown_connections 2023-07-08T12:38:43.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.868+0000 7f50d96c1700 1 --2- 172.21.15.12:0/1373899365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50bc005320 0x7f50bc00ba10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.868+0000 7f50d96c1700 1 -- 172.21.15.12:0/1373899365 >> 172.21.15.12:0/1373899365 conn(0x7f50d4071660 msgr2=0x7f50d4071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:43.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.868+0000 7f50d96c1700 1 -- 172.21.15.12:0/1373899365 shutdown_connections 2023-07-08T12:38:43.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.868+0000 7f50d96c1700 1 -- 172.21.15.12:0/1373899365 wait complete. 2023-07-08T12:38:43.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.870+0000 7f50d96c1700 1 Processor -- start 2023-07-08T12:38:43.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.870+0000 7f50d96c1700 1 -- start start 2023-07-08T12:38:43.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.870+0000 7f50d96c1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50bc005320 0x7f50d40c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.870+0000 7f50d96c1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50d414cc00 0x7f50d40c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.870+0000 7f50d96c1700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50d415c830 0x7f50d41600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.870+0000 7f50d96c1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d414d070 con 0x7f50bc005320 2023-07-08T12:38:43.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.870+0000 7f50d96c1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40c7600 con 0x7f50d414cc00 2023-07-08T12:38:43.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.870+0000 7f50d96c1700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40c7860 con 0x7f50d415c830 2023-07-08T12:38:43.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.870+0000 7f50d2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50bc005320 0x7f50d40c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.870+0000 7f50d2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50bc005320 0x7f50d40c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50230/0 (socket says 172.21.15.12:50230) 2023-07-08T12:38:43.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.870+0000 7f50d2ffd700 1 -- 172.21.15.12:0/3958168667 learned_addr learned my addr 172.21.15.12:0/3958168667 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:43.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.871+0000 7f50d27fc700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50d414cc00 0x7f50d40c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.871+0000 7f50d37fe700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50d415c830 0x7f50d41600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.871+0000 7f50d2ffd700 1 -- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50d415c830 msgr2=0x7f50d41600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.871+0000 7f50d2ffd700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50d415c830 0x7f50d41600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.871+0000 7f50d2ffd700 1 -- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50d414cc00 msgr2=0x7f50d40c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:43.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.871+0000 7f50d2ffd700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50d414cc00 0x7f50d40c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:43.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.871+0000 7f50d2ffd700 1 -- 172.21.15.12:0/3958168667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d414eac0 con 0x7f50bc005320 2023-07-08T12:38:43.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.871+0000 7f50d2ffd700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50bc005320 0x7f50d40c9210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50c80119a0 tx=0x7f50c8011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:43.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.871+0000 7f50bb7fe700 1 -- 172.21.15.12:0/3958168667 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c800f780 con 0x7f50bc005320 2023-07-08T12:38:43.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.871+0000 7f50bb7fe700 1 -- 172.21.15.12:0/3958168667 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c800f8e0 con 0x7f50bc005320 2023-07-08T12:38:43.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.871+0000 7f50d96c1700 1 -- 172.21.15.12:0/3958168667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d414eda0 con 0x7f50bc005320 2023-07-08T12:38:43.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.871+0000 7f50bb7fe700 1 -- 172.21.15.12:0/3958168667 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c800fc70 con 0x7f50bc005320 2023-07-08T12:38:43.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.871+0000 7f50d96c1700 1 -- 172.21.15.12:0/3958168667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d414f360 con 0x7f50bc005320 2023-07-08T12:38:43.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.872+0000 7f50d96c1700 1 -- 172.21.15.12:0/3958168667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d404ea60 con 0x7f50bc005320 2023-07-08T12:38:43.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.873+0000 7f50bb7fe700 1 -- 172.21.15.12:0/3958168667 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f50c801d400 con 0x7f50bc005320 2023-07-08T12:38:43.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.873+0000 7f50bb7fe700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f50c006ace0 0x7f50c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:43.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.873+0000 7f50bb7fe700 1 -- 172.21.15.12:0/3958168667 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f50c8089ce0 con 0x7f50bc005320 2023-07-08T12:38:43.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.873+0000 7f50d27fc700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f50c006ace0 0x7f50c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:43.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.874+0000 7f50d27fc700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f50c006ace0 0x7f50c006d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f50d40ccd50 tx=0x7f50c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:43.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:43.877+0000 7f50bb7fe700 1 -- 172.21.15.12:0/3958168667 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f50c80585c0 con 0x7f50bc005320 2023-07-08T12:38:44.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.119+0000 7f50d96c1700 1 -- 172.21.15.12:0/3958168667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50d40cf690 con 0x7f50bc005320 2023-07-08T12:38:44.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.120+0000 7f50bb7fe700 1 -- 172.21.15.12:0/3958168667 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f50c8058150 con 0x7f50bc005320 2023-07-08T12:38:44.123 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:44.123 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3238006310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3238006310},{"type":"v1","addr":"172.21.15.160:6836","nonce":3238006310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3864661454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3864661454},{"type":"v1","addr":"172.21.15.160:6837","nonce":3864661454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6834/946118830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":946118830},{"type":"v1","addr":"172.21.15.12:6834","nonce":946118830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-08T12:38:07.943202+0000","modified":"2023-07-08T12:38:32.319568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6835/3301134992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3301134992},{"type":"v1","addr":"172.21.15.12:6835","nonce":3301134992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-08T12:38:44.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.121+0000 7f50d96c1700 1 -- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f50c006ace0 msgr2=0x7f50c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.121+0000 7f50d96c1700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f50c006ace0 0x7f50c006d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f50d40ccd50 tx=0x7f50c4009040 comp rx=0 tx=0).stop 2023-07-08T12:38:44.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.122+0000 7f50d96c1700 1 -- 172.21.15.12:0/3958168667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50bc005320 msgr2=0x7f50d40c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.122+0000 7f50d96c1700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50bc005320 0x7f50d40c9210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f50c80119a0 tx=0x7f50c8011cb0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.122+0000 7f50d96c1700 1 -- 172.21.15.12:0/3958168667 shutdown_connections 2023-07-08T12:38:44.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.122+0000 7f50d96c1700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f50c006ace0 0x7f50c006d190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.122+0000 7f50d96c1700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50d415c830 0x7f50d41600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.122+0000 7f50d96c1700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f50bc005320 0x7f50d40c9210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.122+0000 7f50d96c1700 1 --2- 172.21.15.12:0/3958168667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50d414cc00 0x7f50d40c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.122+0000 7f50d96c1700 1 -- 172.21.15.12:0/3958168667 >> 172.21.15.12:0/3958168667 conn(0x7f50d4071660 msgr2=0x7f50d40b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:44.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.122+0000 7f50d96c1700 1 -- 172.21.15.12:0/3958168667 shutdown_connections 2023-07-08T12:38:44.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.122+0000 7f50d96c1700 1 -- 172.21.15.12:0/3958168667 wait complete. 2023-07-08T12:38:44.129 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2023-07-08T12:38:44.137 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:44.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.260+0000 7faf15283700 1 Processor -- start 2023-07-08T12:38:44.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.260+0000 7faf15283700 1 -- start start 2023-07-08T12:38:44.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.261+0000 7faf15283700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10159c30 0x7faf100cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:44.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.261+0000 7faf15283700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf10069f30 0x7faf1006a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:44.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.261+0000 7faf15283700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf1015c890 con 0x7faf10160790 2023-07-08T12:38:44.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.261+0000 7faf15283700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf1015c730 con 0x7faf10069f30 2023-07-08T12:38:44.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.261+0000 7faf15283700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf100cf590 con 0x7faf10159c30 2023-07-08T12:38:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.261+0000 7faf0effd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10159c30 0x7faf100cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.261+0000 7faf0f7fe700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf10069f30 0x7faf1006a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.262+0000 7faf0e7fc700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7faf10160790 0x7faf1006ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:46856/0 (socket says 172.21.15.12:46856) 2023-07-08T12:38:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.262+0000 7faf0e7fc700 1 -- 172.21.15.12:0/1109577765 learned_addr learned my addr 172.21.15.12:0/1109577765 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.262+0000 7faf0dffb700 1 -- 172.21.15.12:0/1109577765 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358987357 0 0) 0x7faf1015c890 con 0x7faf10160790 2023-07-08T12:38:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.262+0000 7faf0f7fe700 1 -- 172.21.15.12:0/1109577765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10159c30 msgr2=0x7faf100cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.262+0000 7faf0f7fe700 1 --2- 172.21.15.12:0/1109577765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10159c30 0x7faf100cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.262+0000 7faf0f7fe700 1 -- 172.21.15.12:0/1109577765 >> v1:172.21.15.160:6789/0 conn(0x7faf10160790 legacy=0x7faf1006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.262+0000 7faf0f7fe700 1 -- 172.21.15.12:0/1109577765 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1006f620 con 0x7faf10069f30 2023-07-08T12:38:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.262+0000 7faf0f7fe700 1 --2- 172.21.15.12:0/1109577765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf10069f30 0x7faf1006a3a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faf08009dc0 tx=0x7faf0800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65eda42711b31912 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.262+0000 7faf0dffb700 1 -- 172.21.15.12:0/1109577765 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0800e070 con 0x7faf10069f30 2023-07-08T12:38:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.262+0000 7faf0dffb700 1 -- 172.21.15.12:0/1109577765 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0800bae0 con 0x7faf10069f30 2023-07-08T12:38:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.262+0000 7faf0dffb700 1 -- 172.21.15.12:0/1109577765 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0800be90 con 0x7faf10069f30 2023-07-08T12:38:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.263+0000 7faf15283700 1 -- 172.21.15.12:0/1109577765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf10069f30 msgr2=0x7faf1006a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.263+0000 7faf15283700 1 --2- 172.21.15.12:0/1109577765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf10069f30 0x7faf1006a3a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faf08009dc0 tx=0x7faf0800b330 comp rx=0 tx=0).stop 2023-07-08T12:38:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.263+0000 7faf15283700 1 -- 172.21.15.12:0/1109577765 shutdown_connections 2023-07-08T12:38:44.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.263+0000 7faf15283700 1 --2- 172.21.15.12:0/1109577765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf10069f30 0x7faf1006a3a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.263+0000 7faf15283700 1 --2- 172.21.15.12:0/1109577765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10159c30 0x7faf100cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.263+0000 7faf15283700 1 -- 172.21.15.12:0/1109577765 >> 172.21.15.12:0/1109577765 conn(0x7faf100cc290 msgr2=0x7faf100cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:44.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.263+0000 7faf15283700 1 -- 172.21.15.12:0/1109577765 shutdown_connections 2023-07-08T12:38:44.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.263+0000 7faf15283700 1 -- 172.21.15.12:0/1109577765 wait complete. 2023-07-08T12:38:44.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.264+0000 7faf15283700 1 Processor -- start 2023-07-08T12:38:44.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.264+0000 7faf15283700 1 -- start start 2023-07-08T12:38:44.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf15283700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf10069f30 0x7faf10156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:44.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf15283700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf10159c30 0x7faf100acb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:44.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf15283700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10160790 0x7faf101e7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:44.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf15283700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf100ad4f0 con 0x7faf10159c30 2023-07-08T12:38:44.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf15283700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf100b55e0 con 0x7faf10069f30 2023-07-08T12:38:44.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf15283700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf100ae7c0 con 0x7faf10160790 2023-07-08T12:38:44.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf0e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf10159c30 0x7faf100acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:44.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf0e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf10159c30 0x7faf100acb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50246/0 (socket says 172.21.15.12:50246) 2023-07-08T12:38:44.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf0e7fc700 1 -- 172.21.15.12:0/192496001 learned_addr learned my addr 172.21.15.12:0/192496001 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:44.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf0f7fe700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10160790 0x7faf101e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:44.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf0effd700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf10069f30 0x7faf10156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:44.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf0e7fc700 1 -- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10160790 msgr2=0x7faf101e7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf0e7fc700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10160790 0x7faf101e7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf0e7fc700 1 -- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf10069f30 msgr2=0x7faf10156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf0e7fc700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf10069f30 0x7faf10156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.265+0000 7faf0e7fc700 1 -- 172.21.15.12:0/192496001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf08009a70 con 0x7faf10159c30 2023-07-08T12:38:44.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.266+0000 7faf0e7fc700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf10159c30 0x7faf100acb60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faf0000c540 tx=0x7faf0000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:44.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.266+0000 7faef77fe700 1 -- 172.21.15.12:0/192496001 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00004c80 con 0x7faf10159c30 2023-07-08T12:38:44.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.266+0000 7faef77fe700 1 -- 172.21.15.12:0/192496001 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf00004de0 con 0x7faf10159c30 2023-07-08T12:38:44.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.266+0000 7faf15283700 1 -- 172.21.15.12:0/192496001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf100ad720 con 0x7faf10159c30 2023-07-08T12:38:44.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.266+0000 7faef77fe700 1 -- 172.21.15.12:0/192496001 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00007310 con 0x7faf10159c30 2023-07-08T12:38:44.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.266+0000 7faf15283700 1 -- 172.21.15.12:0/192496001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf100adce0 con 0x7faf10159c30 2023-07-08T12:38:44.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.267+0000 7faf15283700 1 -- 172.21.15.12:0/192496001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf100cbb40 con 0x7faf10159c30 2023-07-08T12:38:44.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.267+0000 7faef77fe700 1 -- 172.21.15.12:0/192496001 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faf00007470 con 0x7faf10159c30 2023-07-08T12:38:44.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.268+0000 7faef77fe700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faefc06ace0 0x7faefc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:44.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.268+0000 7faef77fe700 1 -- 172.21.15.12:0/192496001 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faf00089410 con 0x7faf10159c30 2023-07-08T12:38:44.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.268+0000 7faf0effd700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faefc06ace0 0x7faefc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:44.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.268+0000 7faf0effd700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faefc06ace0 0x7faefc06d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faf100b2de0 tx=0x7faf04008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:44.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.272+0000 7faef77fe700 1 -- 172.21.15.12:0/192496001 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faf00054260 con 0x7faf10159c30 2023-07-08T12:38:44.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.471+0000 7faf15283700 1 -- 172.21.15.12:0/192496001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf1006a6c0 con 0x7faf10159c30 2023-07-08T12:38:44.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.472+0000 7faef77fe700 1 -- 172.21.15.12:0/192496001 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20108 (secure 0 0 0) 0x7faf00057880 con 0x7faf10159c30 2023-07-08T12:38:44.475 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:44.476 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:12.993803+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:44.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.474+0000 7faf15283700 1 -- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faefc06ace0 msgr2=0x7faefc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.474+0000 7faf15283700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faefc06ace0 0x7faefc06d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faf100b2de0 tx=0x7faf04008040 comp rx=0 tx=0).stop 2023-07-08T12:38:44.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.474+0000 7faf15283700 1 -- 172.21.15.12:0/192496001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf10159c30 msgr2=0x7faf100acb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.474+0000 7faf15283700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf10159c30 0x7faf100acb60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faf0000c540 tx=0x7faf0000c900 comp rx=0 tx=0).stop 2023-07-08T12:38:44.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.474+0000 7faf15283700 1 -- 172.21.15.12:0/192496001 shutdown_connections 2023-07-08T12:38:44.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.474+0000 7faf15283700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7faefc06ace0 0x7faefc06d190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.474+0000 7faf15283700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10160790 0x7faf101e7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.474+0000 7faf15283700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf10069f30 0x7faf10156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.474+0000 7faf15283700 1 --2- 172.21.15.12:0/192496001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf10159c30 0x7faf100acb60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.474+0000 7faf15283700 1 -- 172.21.15.12:0/192496001 >> 172.21.15.12:0/192496001 conn(0x7faf100cc290 msgr2=0x7faf1015bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:44.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.475+0000 7faf15283700 1 -- 172.21.15.12:0/192496001 shutdown_connections 2023-07-08T12:38:44.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.475+0000 7faf15283700 1 -- 172.21.15.12:0/192496001 wait complete. 2023-07-08T12:38:44.490 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-08T12:38:44.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.616+0000 7f4b7ad22700 1 Processor -- start 2023-07-08T12:38:44.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.616+0000 7f4b7ad22700 1 -- start start 2023-07-08T12:38:44.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.617+0000 7f4b7ad22700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4b74159130 0x7f4b740adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:44.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.617+0000 7f4b7ad22700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4b740ae730 0x7f4b740cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:44.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.617+0000 7f4b7ad22700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b740af750 con 0x7f4b740cd960 2023-07-08T12:38:44.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.617+0000 7f4b7ad22700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b740ae5a0 con 0x7f4b74159130 2023-07-08T12:38:44.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.617+0000 7f4b7ad22700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b740d0760 con 0x7f4b740ae730 2023-07-08T12:38:44.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.617+0000 7f4b78abe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4b74159130 0x7f4b740adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:44.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.617+0000 7f4b78abe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4b74159130 0x7f4b740adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50252/0 (socket says 172.21.15.12:50252) 2023-07-08T12:38:44.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.617+0000 7f4b78abe700 1 -- 172.21.15.12:0/1253908022 learned_addr learned my addr 172.21.15.12:0/1253908022 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:44.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.618+0000 7f4b73fff700 1 --2- 172.21.15.12:0/1253908022 >> v2:172.21.15.160:3300/0 conn(0x7f4b740ae730 0x7f4b740cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:44.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.618+0000 7f4b78abe700 1 -- 172.21.15.12:0/1253908022 >> v2:172.21.15.160:3300/0 conn(0x7f4b740ae730 msgr2=0x7f4b740cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:44.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.618+0000 7f4b78abe700 1 --2- 172.21.15.12:0/1253908022 >> v2:172.21.15.160:3300/0 conn(0x7f4b740ae730 0x7f4b740cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.618+0000 7f4b78abe700 1 -- 172.21.15.12:0/1253908022 >> v1:172.21.15.160:6789/0 conn(0x7f4b740cd960 legacy=0x7f4b740ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.618+0000 7f4b78abe700 1 -- 172.21.15.12:0/1253908022 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b740cdd80 con 0x7f4b74159130 2023-07-08T12:38:44.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.618+0000 7f4b73fff700 1 --2- 172.21.15.12:0/1253908022 >> v2:172.21.15.160:3300/0 conn(0x7f4b740ae730 0x7f4b740cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:44.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.618+0000 7f4b78abe700 1 --2- 172.21.15.12:0/1253908022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b74159130 0x7f4b740adea0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b74071a20 tx=0x7f4b6800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53fbd8ae26af090a server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:44.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.618+0000 7f4b737fe700 1 -- 172.21.15.12:0/1253908022 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6800dd30 con 0x7f4b74159130 2023-07-08T12:38:44.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.618+0000 7f4b737fe700 1 -- 172.21.15.12:0/1253908022 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6800de90 con 0x7f4b74159130 2023-07-08T12:38:44.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.618+0000 7f4b737fe700 1 -- 172.21.15.12:0/1253908022 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68010690 con 0x7f4b74159130 2023-07-08T12:38:44.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.619+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1253908022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b74159130 msgr2=0x7f4b740adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.619+0000 7f4b7ad22700 1 --2- 172.21.15.12:0/1253908022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b74159130 0x7f4b740adea0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b74071a20 tx=0x7f4b6800be70 comp rx=0 tx=0).stop 2023-07-08T12:38:44.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.619+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1253908022 shutdown_connections 2023-07-08T12:38:44.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.619+0000 7f4b7ad22700 1 --2- 172.21.15.12:0/1253908022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b74159130 0x7f4b740adea0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.619+0000 7f4b7ad22700 1 --2- 172.21.15.12:0/1253908022 >> v2:172.21.15.160:3300/0 conn(0x7f4b740ae730 0x7f4b740cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.619+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1253908022 >> 172.21.15.12:0/1253908022 conn(0x7f4b74098a30 msgr2=0x7f4b740cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:44.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.619+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1253908022 shutdown_connections 2023-07-08T12:38:44.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.619+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1253908022 wait complete. 2023-07-08T12:38:44.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.620+0000 7f4b7ad22700 1 Processor -- start 2023-07-08T12:38:44.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.620+0000 7f4b7ad22700 1 -- start start 2023-07-08T12:38:44.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b7ad22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b740ae730 0x7f4b740b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:44.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b7ad22700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b740cd960 0x7f4b74149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:44.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b7ad22700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b74159130 0x7f4b740b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:44.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b7ad22700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740b5730 con 0x7f4b740ae730 2023-07-08T12:38:44.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b7ad22700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740b2c60 con 0x7f4b74159130 2023-07-08T12:38:44.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b7ad22700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740b5170 con 0x7f4b740cd960 2023-07-08T12:38:44.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b78abe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b740ae730 0x7f4b740b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:44.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b78abe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b740ae730 0x7f4b740b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50260/0 (socket says 172.21.15.12:50260) 2023-07-08T12:38:44.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b78abe700 1 -- 172.21.15.12:0/1405717615 learned_addr learned my addr 172.21.15.12:0/1405717615 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:44.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b792bf700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b74159130 0x7f4b740b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:44.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b73fff700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b740cd960 0x7f4b74149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:44.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b78abe700 1 -- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b740cd960 msgr2=0x7f4b74149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b78abe700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b740cd960 0x7f4b74149b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b78abe700 1 -- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b74159130 msgr2=0x7f4b740b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b78abe700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b74159130 0x7f4b740b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b78abe700 1 -- 172.21.15.12:0/1405717615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6800b7f0 con 0x7f4b740ae730 2023-07-08T12:38:44.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.621+0000 7f4b78abe700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b740ae730 0x7f4b740b2680 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800b7c0 tx=0x7f4b68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:44.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.622+0000 7f4b717fa700 1 -- 172.21.15.12:0/1405717615 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6801d070 con 0x7f4b740ae730 2023-07-08T12:38:44.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.622+0000 7f4b717fa700 1 -- 172.21.15.12:0/1405717615 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6801e040 con 0x7f4b740ae730 2023-07-08T12:38:44.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.622+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1405717615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b740b5450 con 0x7f4b740ae730 2023-07-08T12:38:44.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.622+0000 7f4b717fa700 1 -- 172.21.15.12:0/1405717615 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68010e50 con 0x7f4b740ae730 2023-07-08T12:38:44.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.622+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1405717615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7414a0a0 con 0x7f4b740ae730 2023-07-08T12:38:44.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.623+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1405717615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7404ea60 con 0x7f4b740ae730 2023-07-08T12:38:44.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.623+0000 7f4b717fa700 1 -- 172.21.15.12:0/1405717615 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4b680036a0 con 0x7f4b740ae730 2023-07-08T12:38:44.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.623+0000 7f4b717fa700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4b5806ace0 0x7f4b5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:44.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.624+0000 7f4b717fa700 1 -- 172.21.15.12:0/1405717615 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4b6808d7d0 con 0x7f4b740ae730 2023-07-08T12:38:44.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.624+0000 7f4b73fff700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4b5806ace0 0x7f4b5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:44.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.624+0000 7f4b73fff700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4b5806ace0 0x7f4b5806d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b741f1130 tx=0x7f4b60008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:44.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.628+0000 7f4b717fa700 1 -- 172.21.15.12:0/1405717615 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4b68058740 con 0x7f4b740ae730 2023-07-08T12:38:44.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.874+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1405717615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b740af3d0 con 0x7f4b740ae730 2023-07-08T12:38:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.875+0000 7f4b717fa700 1 -- 172.21.15.12:0/1405717615 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f4b6805bd60 con 0x7f4b740ae730 2023-07-08T12:38:44.878 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:44.878 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3238006310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3238006310},{"type":"v1","addr":"172.21.15.160:6836","nonce":3238006310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3864661454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3864661454},{"type":"v1","addr":"172.21.15.160:6837","nonce":3864661454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6834/946118830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":946118830},{"type":"v1","addr":"172.21.15.12:6834","nonce":946118830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-08T12:38:07.943202+0000","modified":"2023-07-08T12:38:32.319568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6835/3301134992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3301134992},{"type":"v1","addr":"172.21.15.12:6835","nonce":3301134992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-08T12:38:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.877+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4b5806ace0 msgr2=0x7f4b5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.877+0000 7f4b7ad22700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4b5806ace0 0x7f4b5806d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b741f1130 tx=0x7f4b60008040 comp rx=0 tx=0).stop 2023-07-08T12:38:44.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.877+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1405717615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b740ae730 msgr2=0x7f4b740b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:44.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.877+0000 7f4b7ad22700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b740ae730 0x7f4b740b2680 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800b7c0 tx=0x7f4b68004750 comp rx=0 tx=0).stop 2023-07-08T12:38:44.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.877+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1405717615 shutdown_connections 2023-07-08T12:38:44.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.877+0000 7f4b7ad22700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4b5806ace0 0x7f4b5806d190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.877+0000 7f4b7ad22700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b74159130 0x7f4b740b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.877+0000 7f4b7ad22700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b740ae730 0x7f4b740b2680 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.877+0000 7f4b7ad22700 1 --2- 172.21.15.12:0/1405717615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b740cd960 0x7f4b74149b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.877+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1405717615 >> 172.21.15.12:0/1405717615 conn(0x7f4b74098a30 msgr2=0x7f4b74069f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.877+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1405717615 shutdown_connections 2023-07-08T12:38:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:44.877+0000 7f4b7ad22700 1 -- 172.21.15.12:0/1405717615 wait complete. 2023-07-08T12:38:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2023-07-08T12:38:45.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-08T12:38:07.943202+0000', 'modified': '2023-07-08T12:38:32.319568+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4290}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4290': {'gid': 4290, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.12:6835/3301134992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6833', 'nonce': 3301134992}, {'type': 'v1', 'addr': '172.21.15.12:6835', 'nonce': 3301134992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-08T12:38:45.295 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-08T12:38:45.295 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-08T12:38:45.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.426+0000 7f475ea45700 1 Processor -- start 2023-07-08T12:38:45.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.426+0000 7f475ea45700 1 -- start start 2023-07-08T12:38:45.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.427+0000 7f475ea45700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f47580cd7a0 0x7f47580cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.427+0000 7f475ea45700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f47580cd1c0 0x7f4758069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.427+0000 7f475ea45700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47581594b0 con 0x7f47580caf90 2023-07-08T12:38:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.427+0000 7f475ea45700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47580ce0b0 con 0x7f47580cd7a0 2023-07-08T12:38:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.427+0000 7f475ea45700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47580ca1d0 con 0x7f47580cd1c0 2023-07-08T12:38:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.427+0000 7f4757fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f47580cd7a0 0x7f47580cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.427+0000 7f4757fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f47580cd7a0 0x7f47580cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50266/0 (socket says 172.21.15.12:50266) 2023-07-08T12:38:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.427+0000 7f4757fff700 1 -- 172.21.15.12:0/738120063 learned_addr learned my addr 172.21.15.12:0/738120063 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:45.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.428+0000 7f475cfe2700 1 --2- 172.21.15.12:0/738120063 >> v2:172.21.15.160:3300/0 conn(0x7f47580cd1c0 0x7f4758069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.428+0000 7f4757fff700 1 -- 172.21.15.12:0/738120063 >> v2:172.21.15.160:3300/0 conn(0x7f47580cd1c0 msgr2=0x7f4758069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.428+0000 7f4757fff700 1 --2- 172.21.15.12:0/738120063 >> v2:172.21.15.160:3300/0 conn(0x7f47580cd1c0 0x7f4758069f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.428+0000 7f4757fff700 1 -- 172.21.15.12:0/738120063 >> v1:172.21.15.160:6789/0 conn(0x7f47580caf90 legacy=0x7f47580ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.428+0000 7f4757fff700 1 -- 172.21.15.12:0/738120063 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47580ca4b0 con 0x7f47580cd7a0 2023-07-08T12:38:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.428+0000 7f4757fff700 1 --2- 172.21.15.12:0/738120063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47580cd7a0 0x7f47580cdb70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f475815dc70 tx=0x7f4748009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e67a5d273d1593f server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.428+0000 7f4756ffd700 1 -- 172.21.15.12:0/738120063 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748004230 con 0x7f47580cd7a0 2023-07-08T12:38:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.428+0000 7f4756ffd700 1 -- 172.21.15.12:0/738120063 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4748004390 con 0x7f47580cd7a0 2023-07-08T12:38:45.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.428+0000 7f4756ffd700 1 -- 172.21.15.12:0/738120063 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47480036a0 con 0x7f47580cd7a0 2023-07-08T12:38:45.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.429+0000 7f475ea45700 1 -- 172.21.15.12:0/738120063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47580cd7a0 msgr2=0x7f47580cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:45.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.429+0000 7f475ea45700 1 --2- 172.21.15.12:0/738120063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47580cd7a0 0x7f47580cdb70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f475815dc70 tx=0x7f4748009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.429+0000 7f475ea45700 1 -- 172.21.15.12:0/738120063 shutdown_connections 2023-07-08T12:38:45.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.429+0000 7f475ea45700 1 --2- 172.21.15.12:0/738120063 >> v2:172.21.15.160:3300/0 conn(0x7f47580cd1c0 0x7f4758069f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.429+0000 7f475ea45700 1 --2- 172.21.15.12:0/738120063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47580cd7a0 0x7f47580cdb70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.429+0000 7f475ea45700 1 -- 172.21.15.12:0/738120063 >> 172.21.15.12:0/738120063 conn(0x7f47580cb9f0 msgr2=0x7f47580cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:45.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.429+0000 7f475ea45700 1 -- 172.21.15.12:0/738120063 shutdown_connections 2023-07-08T12:38:45.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.429+0000 7f475ea45700 1 -- 172.21.15.12:0/738120063 wait complete. 2023-07-08T12:38:45.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.430+0000 7f475ea45700 1 Processor -- start 2023-07-08T12:38:45.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.430+0000 7f475ea45700 1 -- start start 2023-07-08T12:38:45.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f475ea45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47580caf90 0x7f47580b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f475ea45700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47580cd1c0 0x7f47580c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f475ea45700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47580cd7a0 0x7f475814c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f475ea45700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47580c5730 con 0x7f47580caf90 2023-07-08T12:38:45.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f475ea45700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47580b3750 con 0x7f47580cd1c0 2023-07-08T12:38:45.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f475ea45700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47580b4c50 con 0x7f47580cd7a0 2023-07-08T12:38:45.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f4757fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47580caf90 0x7f47580b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f4757fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47580caf90 0x7f47580b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50272/0 (socket says 172.21.15.12:50272) 2023-07-08T12:38:45.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f4757fff700 1 -- 172.21.15.12:0/1524873770 learned_addr learned my addr 172.21.15.12:0/1524873770 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:45.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f47577fe700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47580cd1c0 0x7f47580c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f475cfe2700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47580cd7a0 0x7f475814c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f4757fff700 1 -- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47580cd7a0 msgr2=0x7f475814c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:45.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f4757fff700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47580cd7a0 0x7f475814c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f4757fff700 1 -- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47580cd1c0 msgr2=0x7f47580c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:45.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f4757fff700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47580cd1c0 0x7f47580c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f4757fff700 1 -- 172.21.15.12:0/1524873770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4748009550 con 0x7f47580caf90 2023-07-08T12:38:45.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f475cfe2700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47580cd7a0 0x7f475814c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:45.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.431+0000 7f4757fff700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47580caf90 0x7f47580b4710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4748009130 tx=0x7f4748004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:45.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.432+0000 7f4754ff9700 1 -- 172.21.15.12:0/1524873770 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748003e20 con 0x7f47580caf90 2023-07-08T12:38:45.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.432+0000 7f4754ff9700 1 -- 172.21.15.12:0/1524873770 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4748011630 con 0x7f47580caf90 2023-07-08T12:38:45.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.432+0000 7f4754ff9700 1 -- 172.21.15.12:0/1524873770 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47480119a0 con 0x7f47580caf90 2023-07-08T12:38:45.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.432+0000 7f475ea45700 1 -- 172.21.15.12:0/1524873770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47580b2b40 con 0x7f47580caf90 2023-07-08T12:38:45.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.432+0000 7f475ea45700 1 -- 172.21.15.12:0/1524873770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47580b3100 con 0x7f47580caf90 2023-07-08T12:38:45.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.433+0000 7f4754ff9700 1 -- 172.21.15.12:0/1524873770 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4748011b00 con 0x7f47580caf90 2023-07-08T12:38:45.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.433+0000 7f4754ff9700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f473806ad10 0x7f473806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.434+0000 7f4754ff9700 1 -- 172.21.15.12:0/1524873770 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f474808acd0 con 0x7f47580caf90 2023-07-08T12:38:45.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.434+0000 7f47577fe700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f473806ad10 0x7f473806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.434+0000 7f47577fe700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f473806ad10 0x7f473806d1c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4758071f60 tx=0x7f4740007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:45.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.436+0000 7f476abcab80 1 Processor -- start 2023-07-08T12:38:45.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.436+0000 7f476abcab80 1 -- start start 2023-07-08T12:38:45.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f476abcab80 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a1d90 0x55783c35a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f476abcab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 0x55783c384030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f476abcab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c384570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f476abcab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783c25ec60 con 0x55783c395c40 2023-07-08T12:38:45.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f476abcab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783c2b19f0 con 0x55783c3a2160 2023-07-08T12:38:45.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f476abcab80 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55783c2aae80 con 0x55783c3a1d90 2023-07-08T12:38:45.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f475cfe2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c384570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f475cfe2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c384570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50284/0 (socket says 172.21.15.12:50284) 2023-07-08T12:38:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f475cfe2700 1 -- 172.21.15.12:0/1933397317 learned_addr learned my addr 172.21.15.12:0/1933397317 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f4757fff700 1 --2- 172.21.15.12:0/1933397317 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a1d90 0x55783c35a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f47577fe700 1 --2- 172.21.15.12:0/1933397317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 0x55783c384030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f475cfe2700 1 -- 172.21.15.12:0/1933397317 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a1d90 msgr2=0x55783c35a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f475cfe2700 1 --2- 172.21.15.12:0/1933397317 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a1d90 0x55783c35a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f475cfe2700 1 -- 172.21.15.12:0/1933397317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 msgr2=0x55783c384030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:45.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f475cfe2700 1 --2- 172.21.15.12:0/1933397317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 0x55783c384030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f475cfe2700 1 -- 172.21.15.12:0/1933397317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55783c0d8d90 con 0x55783c395c40 2023-07-08T12:38:45.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f4757fff700 1 --2- 172.21.15.12:0/1933397317 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a1d90 0x55783c35a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:45.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f475cfe2700 1 --2- 172.21.15.12:0/1933397317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c384570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4750007510 tx=0x7f4750007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477bcc648361d279 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:45.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f474dffb700 1 -- 172.21.15.12:0/1933397317 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47500147e0 con 0x55783c395c40 2023-07-08T12:38:45.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f474dffb700 1 -- 172.21.15.12:0/1933397317 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475000f050 con 0x55783c395c40 2023-07-08T12:38:45.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.437+0000 7f474dffb700 1 -- 172.21.15.12:0/1933397317 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4750014b40 con 0x55783c395c40 2023-07-08T12:38:45.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.438+0000 7f476abcab80 1 -- 172.21.15.12:0/1933397317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 msgr2=0x55783c384570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:45.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.438+0000 7f476abcab80 1 --2- 172.21.15.12:0/1933397317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c384570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4750007510 tx=0x7f4750007820 comp rx=0 tx=0).stop 2023-07-08T12:38:45.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.438+0000 7f476abcab80 1 -- 172.21.15.12:0/1933397317 shutdown_connections 2023-07-08T12:38:45.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.438+0000 7f476abcab80 1 --2- 172.21.15.12:0/1933397317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c384570 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.438+0000 7f476abcab80 1 --2- 172.21.15.12:0/1933397317 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a1d90 0x55783c35a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.438+0000 7f476abcab80 1 --2- 172.21.15.12:0/1933397317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 0x55783c384030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.438+0000 7f476abcab80 1 -- 172.21.15.12:0/1933397317 >> 172.21.15.12:0/1933397317 conn(0x55783c34ea90 msgr2=0x55783c354010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:45.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.438+0000 7f476abcab80 1 -- 172.21.15.12:0/1933397317 shutdown_connections 2023-07-08T12:38:45.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.438+0000 7f476abcab80 1 -- 172.21.15.12:0/1933397317 wait complete. 2023-07-08T12:38:45.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.439+0000 7f476abcab80 1 -- start start 2023-07-08T12:38:45.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.439+0000 7f476abcab80 1 Processor -- start 2023-07-08T12:38:45.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f476abcab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c37b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f476abcab80 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a1d90 0x55783c37fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f476abcab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 0x55783c385fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f476abcab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783c24aa70 con 0x55783c395c40 2023-07-08T12:38:45.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f476abcab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783c24a770 con 0x55783c3a2160 2023-07-08T12:38:45.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f476abcab80 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55783c12e8d0 con 0x55783c3a1d90 2023-07-08T12:38:45.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f476abcab80 10 client.0 _ll_register_callbacks cb 0x55783c248710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-08T12:38:45.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f4757fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c37b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f4757fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c37b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50286/0 (socket says 172.21.15.12:50286) 2023-07-08T12:38:45.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f4757fff700 1 -- 172.21.15.12:0/138133208 learned_addr learned my addr 172.21.15.12:0/138133208 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:45.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f475cfe2700 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 0x55783c385fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f47577fe700 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a1d90 0x55783c37fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f4757fff700 1 -- 172.21.15.12:0/138133208 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a1d90 msgr2=0x55783c37fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:45.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f4757fff700 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a1d90 0x55783c37fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f4757fff700 1 -- 172.21.15.12:0/138133208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 msgr2=0x55783c385fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:45.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f4757fff700 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 0x55783c385fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f4757fff700 1 -- 172.21.15.12:0/138133208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4750009d50 con 0x55783c395c40 2023-07-08T12:38:45.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f47577fe700 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a1d90 0x55783c37fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:45.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f4757fff700 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c37b480 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4748088e40 tx=0x7f474805d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0da882592d9e1a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:45.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f4736ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-07-08T12:38:45.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f4736ffd700 1 -- 172.21.15.12:0/138133208 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748006ea0 con 0x55783c395c40 2023-07-08T12:38:45.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f4736ffd700 1 -- 172.21.15.12:0/138133208 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4748055e70 con 0x55783c395c40 2023-07-08T12:38:45.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f476abcab80 1 -- 172.21.15.12:0/138133208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55783c34c0a0 con 0x55783c395c40 2023-07-08T12:38:45.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f4736ffd700 1 -- 172.21.15.12:0/138133208 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4748072020 con 0x55783c395c40 2023-07-08T12:38:45.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f476abcab80 10 client.4839 fetch_fsmap learned FSMap version 9 2023-07-08T12:38:45.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.440+0000 7f476abcab80 1 -- 172.21.15.12:0/138133208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55783c0c1c10 con 0x55783c395c40 2023-07-08T12:38:45.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.441+0000 7f4736ffd700 1 -- 172.21.15.12:0/138133208 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55783c34c0a0 con 0x55783c395c40 2023-07-08T12:38:45.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.441+0000 7f4736ffd700 1 -- 172.21.15.12:0/138133208 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f4748058c50 con 0x55783c395c40 2023-07-08T12:38:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.441+0000 7f476abcab80 10 client.4839 fetch_fsmap finished waiting for FSMap version 9 2023-07-08T12:38:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.441+0000 7f476abcab80 10 client.4839 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-08T12:38:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.441+0000 7f476abcab80 20 client.4839 populate_metadata read hostname 'smithi012' 2023-07-08T12:38:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.441+0000 7f476abcab80 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x55783c3a0160 0x55783c3a0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.441+0000 7f476abcab80 4 client.4839 mds_command: new command op to 4290 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-08T12:38:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.441+0000 7f476abcab80 1 -- 172.21.15.12:0/138133208 --> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55783c2a3e60 con 0x55783c3a0160 2023-07-08T12:38:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.441+0000 7f47577fe700 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x55783c3a0160 0x55783c3a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.441+0000 7f47577fe700 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x55783c3a0160 0x55783c3a0530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce9e6c8ecb6e698 server_cookie=fb9d145e1cd8130c in_seq=0 out_seq=0 2023-07-08T12:38:45.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.441+0000 7f4736ffd700 10 client.4839 ms_handle_connect on v2:172.21.15.12:6833/3301134992 2023-07-08T12:38:45.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f4736ffd700 1 -- 172.21.15.12:0/138133208 <== mds.0 v2:172.21.15.12:6833/3301134992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55783c2a3e60 con 0x55783c3a0160 2023-07-08T12:38:45.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f4736ffd700 10 client.4839 handle_command_reply: tid=0 2023-07-08T12:38:45.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f476abcab80 1 client.4839 shutdown 2023-07-08T12:38:45.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f476abcab80 1 -- 172.21.15.12:0/138133208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 msgr2=0x55783c37b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:45.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f476abcab80 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c37b480 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4748088e40 tx=0x7f474805d720 comp rx=0 tx=0).stop 2023-07-08T12:38:45.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f476abcab80 1 -- 172.21.15.12:0/138133208 shutdown_connections 2023-07-08T12:38:45.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f476abcab80 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 0x55783c385fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f476abcab80 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x55783c3a0160 0x55783c3a0530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f476abcab80 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c37b480 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f476abcab80 1 --2- 172.21.15.12:0/138133208 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a1d90 0x55783c37fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f4736ffd700 0 client.4839 ms_handle_reset on v2:172.21.15.12:6833/3301134992 2023-07-08T12:38:45.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f476abcab80 1 -- 172.21.15.12:0/138133208 >> 172.21.15.12:0/138133208 conn(0x55783c34ea90 msgr2=0x55783c3592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:45.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f476abcab80 1 -- 172.21.15.12:0/138133208 shutdown_connections 2023-07-08T12:38:45.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f476abcab80 1 -- 172.21.15.12:0/138133208 wait complete. 2023-07-08T12:38:45.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.442+0000 7f476abcab80 20 client.4839 trim_cache size 0 max 16384 2023-07-08T12:38:45.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.460+0000 7f476abcab80 1 Processor -- start 2023-07-08T12:38:45.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.460+0000 7f476abcab80 1 -- start start 2023-07-08T12:38:45.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f476abcab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c387eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f476abcab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a1d90 0x55783c398360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f476abcab80 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a2160 0x55783c354cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f476abcab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783c3b8380 con 0x55783c395c40 2023-07-08T12:38:45.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f476abcab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783c354a50 con 0x55783c3a1d90 2023-07-08T12:38:45.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f476abcab80 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55783c3c0fd0 con 0x55783c3a2160 2023-07-08T12:38:45.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f4757fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c387eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f4757fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c387eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50294/0 (socket says 172.21.15.12:50294) 2023-07-08T12:38:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f4757fff700 1 -- 172.21.15.12:0/3640127519 learned_addr learned my addr 172.21.15.12:0/3640127519 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f475cfe2700 1 --2- 172.21.15.12:0/3640127519 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a2160 0x55783c354cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f4757fff700 1 -- 172.21.15.12:0/3640127519 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a2160 msgr2=0x55783c354cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f47577fe700 1 --2- 172.21.15.12:0/3640127519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a1d90 0x55783c398360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f4757fff700 1 --2- 172.21.15.12:0/3640127519 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a2160 0x55783c354cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f4757fff700 1 -- 172.21.15.12:0/3640127519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a1d90 msgr2=0x55783c398360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:45.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f4757fff700 1 --2- 172.21.15.12:0/3640127519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a1d90 0x55783c398360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.461+0000 7f4757fff700 1 -- 172.21.15.12:0/3640127519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55783c0ff200 con 0x55783c395c40 2023-07-08T12:38:45.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.462+0000 7f4757fff700 1 --2- 172.21.15.12:0/3640127519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c387eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f474800c010 tx=0x7f4748078050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e7b3d732045a96 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:45.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.462+0000 7f474e7fc700 1 -- 172.21.15.12:0/3640127519 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748072030 con 0x55783c395c40 2023-07-08T12:38:45.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.462+0000 7f474e7fc700 1 -- 172.21.15.12:0/3640127519 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47480592c0 con 0x55783c395c40 2023-07-08T12:38:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.462+0000 7f474e7fc700 1 -- 172.21.15.12:0/3640127519 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748055e70 con 0x55783c395c40 2023-07-08T12:38:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.462+0000 7f476abcab80 1 -- 172.21.15.12:0/3640127519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 msgr2=0x55783c387eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.462+0000 7f476abcab80 1 --2- 172.21.15.12:0/3640127519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c387eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f474800c010 tx=0x7f4748078050 comp rx=0 tx=0).stop 2023-07-08T12:38:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.462+0000 7f476abcab80 1 -- 172.21.15.12:0/3640127519 shutdown_connections 2023-07-08T12:38:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.462+0000 7f476abcab80 1 --2- 172.21.15.12:0/3640127519 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c3a2160 0x55783c354cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.462+0000 7f476abcab80 1 --2- 172.21.15.12:0/3640127519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c395c40 0x55783c387eb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.462+0000 7f476abcab80 1 --2- 172.21.15.12:0/3640127519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a1d90 0x55783c398360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.462+0000 7f476abcab80 1 -- 172.21.15.12:0/3640127519 >> 172.21.15.12:0/3640127519 conn(0x55783c34ea90 msgr2=0x55783c350000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:45.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.462+0000 7f476abcab80 1 -- 172.21.15.12:0/3640127519 shutdown_connections 2023-07-08T12:38:45.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.462+0000 7f476abcab80 1 -- 172.21.15.12:0/3640127519 wait complete. 2023-07-08T12:38:45.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.463+0000 7f476abcab80 1 -- start start 2023-07-08T12:38:45.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.463+0000 7f476abcab80 1 Processor -- start 2023-07-08T12:38:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f476abcab80 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c395c40 0x55783c353830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f476abcab80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c3a1d90 0x55783c404bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f476abcab80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 0x55783c394e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f476abcab80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783c405590 con 0x55783c3a1d90 2023-07-08T12:38:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f476abcab80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783c353e10 con 0x55783c3a2160 2023-07-08T12:38:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f476abcab80 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55783c405100 con 0x55783c395c40 2023-07-08T12:38:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f476abcab80 10 client.0 _ll_register_callbacks cb 0x7f4748011db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-08T12:38:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f47577fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c3a1d90 0x55783c404bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f47577fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c3a1d90 0x55783c404bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50302/0 (socket says 172.21.15.12:50302) 2023-07-08T12:38:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f47577fe700 1 -- 172.21.15.12:0/2166929811 learned_addr learned my addr 172.21.15.12:0/2166929811 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f475cfe2700 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 0x55783c394e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f4757fff700 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c395c40 0x55783c353830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f47577fe700 1 -- 172.21.15.12:0/2166929811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c395c40 msgr2=0x55783c353830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f47577fe700 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c395c40 0x55783c353830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f47577fe700 1 -- 172.21.15.12:0/2166929811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 msgr2=0x55783c394e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f47577fe700 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 0x55783c394e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f47577fe700 1 -- 172.21.15.12:0/2166929811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4750009d50 con 0x55783c3a1d90 2023-07-08T12:38:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f4757fff700 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c395c40 0x55783c353830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f47577fe700 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c3a1d90 0x55783c404bc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4740003da0 tx=0x7f4740003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32de75808a6f701f server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f4736ffd700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-07-08T12:38:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f4736ffd700 1 -- 172.21.15.12:0/2166929811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474001d2e0 con 0x55783c3a1d90 2023-07-08T12:38:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f4736ffd700 1 -- 172.21.15.12:0/2166929811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f474002cac0 con 0x55783c3a1d90 2023-07-08T12:38:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.464+0000 7f476abcab80 1 -- 172.21.15.12:0/2166929811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55783c266000 con 0x55783c3a1d90 2023-07-08T12:38:45.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f4736ffd700 1 -- 172.21.15.12:0/2166929811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f474002cc80 con 0x55783c3a1d90 2023-07-08T12:38:45.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f476abcab80 10 client.4845 fetch_fsmap learned FSMap version 9 2023-07-08T12:38:45.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f476abcab80 1 -- 172.21.15.12:0/2166929811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55783c0d7260 con 0x55783c3a1d90 2023-07-08T12:38:45.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f4736ffd700 1 -- 172.21.15.12:0/2166929811 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55783c266000 con 0x55783c3a1d90 2023-07-08T12:38:45.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f4736ffd700 1 -- 172.21.15.12:0/2166929811 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f474002c0e0 con 0x55783c3a1d90 2023-07-08T12:38:45.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f476abcab80 10 client.4845 fetch_fsmap finished waiting for FSMap version 9 2023-07-08T12:38:45.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f476abcab80 10 client.4845 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-08T12:38:45.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f476abcab80 20 client.4845 populate_metadata read hostname 'smithi012' 2023-07-08T12:38:45.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f476abcab80 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x55783c3592f0 0x55783c3596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:45.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f476abcab80 4 client.4845 mds_command: new command op to 4290 tid=0[{"prefix": "status"}] 2023-07-08T12:38:45.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f476abcab80 1 -- 172.21.15.12:0/2166929811 --> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55783c0f9f20 con 0x55783c3592f0 2023-07-08T12:38:45.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f4757fff700 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x55783c3592f0 0x55783c3596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:45.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f4736ffd700 10 client.4845 ms_handle_connect on v2:172.21.15.12:6833/3301134992 2023-07-08T12:38:45.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.465+0000 7f4757fff700 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x55783c3592f0 0x55783c3596c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a28a2a2baf8726fd server_cookie=a374566447d8d9c2 in_seq=0 out_seq=0 2023-07-08T12:38:45.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f4736ffd700 1 -- 172.21.15.12:0/2166929811 <== mds.0 v2:172.21.15.12:6833/3301134992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55783c0f9f20 con 0x55783c3592f0 2023-07-08T12:38:45.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f4736ffd700 10 client.4845 handle_command_reply: tid=0 2023-07-08T12:38:45.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f476abcab80 1 client.4845 shutdown 2023-07-08T12:38:45.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f476abcab80 1 -- 172.21.15.12:0/2166929811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c3a1d90 msgr2=0x55783c404bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:45.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f476abcab80 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c3a1d90 0x55783c404bc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4740003da0 tx=0x7f4740003780 comp rx=0 tx=0).stop 2023-07-08T12:38:45.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f476abcab80 1 -- 172.21.15.12:0/2166929811 shutdown_connections 2023-07-08T12:38:45.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f476abcab80 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55783c3a2160 0x55783c394e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f476abcab80 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55783c395c40 0x55783c353830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f476abcab80 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.12:6833/3301134992,v1:172.21.15.12:6835/3301134992] conn(0x55783c3592f0 0x55783c3596c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f476abcab80 1 --2- 172.21.15.12:0/2166929811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55783c3a1d90 0x55783c404bc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f4736ffd700 0 client.4845 ms_handle_reset on v2:172.21.15.12:6833/3301134992 2023-07-08T12:38:45.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f476abcab80 1 -- 172.21.15.12:0/2166929811 >> 172.21.15.12:0/2166929811 conn(0x55783c34ea90 msgr2=0x55783c354cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:45.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f476abcab80 1 -- 172.21.15.12:0/2166929811 shutdown_connections 2023-07-08T12:38:45.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f476abcab80 1 -- 172.21.15.12:0/2166929811 wait complete. 2023-07-08T12:38:45.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f476abcab80 20 client.4845 trim_cache size 0 max 16384 2023-07-08T12:38:45.481 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-07-08T12:38:45.482 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "71d270f5-58a7-4223-8d07-37436121729a", 2023-07-08T12:38:45.482 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2023-07-08T12:38:45.482 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 4290, 2023-07-08T12:38:45.482 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2023-07-08T12:38:45.482 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2023-07-08T12:38:45.482 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2023-07-08T12:38:45.483 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 36.922186723000003, 2023-07-08T12:38:45.483 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 9, 2023-07-08T12:38:45.483 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 28, 2023-07-08T12:38:45.483 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 28, 2023-07-08T12:38:45.483 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 42.975811606000001 2023-07-08T12:38:45.483 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-07-08T12:38:45.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f475ea45700 1 -- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f473806ad10 msgr2=0x7f473806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:45.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.466+0000 7f475ea45700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f473806ad10 0x7f473806d1c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4758071f60 tx=0x7f4740007ec0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.467+0000 7f475ea45700 1 -- 172.21.15.12:0/1524873770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47580caf90 msgr2=0x7f47580b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:45.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.467+0000 7f475ea45700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47580caf90 0x7f47580b4710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4748009130 tx=0x7f4748004750 comp rx=0 tx=0).stop 2023-07-08T12:38:45.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.467+0000 7f475ea45700 1 -- 172.21.15.12:0/1524873770 shutdown_connections 2023-07-08T12:38:45.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.467+0000 7f475ea45700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f473806ad10 0x7f473806d1c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.467+0000 7f475ea45700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47580cd7a0 0x7f475814c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.467+0000 7f475ea45700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47580caf90 0x7f47580b4710 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.467+0000 7f475ea45700 1 --2- 172.21.15.12:0/1524873770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47580cd1c0 0x7f47580c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:45.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.467+0000 7f475ea45700 1 -- 172.21.15.12:0/1524873770 >> 172.21.15.12:0/1524873770 conn(0x7f47580cb9f0 msgr2=0x7f4758070bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:45.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.467+0000 7f475ea45700 1 -- 172.21.15.12:0/1524873770 shutdown_connections 2023-07-08T12:38:45.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:45.467+0000 7f475ea45700 1 -- 172.21.15.12:0/1524873770 wait complete. 2023-07-08T12:38:45.487 INFO:teuthology.run_tasks:Running task kclient... 2023-07-08T12:38:45.501 INFO:tasks.kclient:Mounting kernel clients... 2023-07-08T12:38:45.501 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-07-08T12:38:45.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:45.502 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2023-07-08T12:38:45.519 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-07-08T12:38:45.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:45.521 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-08T12:38:45.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:45.606 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2023-07-08T12:38:45.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:45.730 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2023-07-08T12:38:45.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:45.749 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2023-07-08T12:38:45.818 INFO:teuthology.orchestra.run.smithi012.stderr:Cannot find device "ceph-brx" 2023-07-08T12:38:45.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-08T12:38:45.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:45.820 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2023-07-08T12:38:45.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:45.844 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link delete ceph-brx 2023-07-08T12:38:45.919 INFO:teuthology.orchestra.run.smithi160.stderr:Cannot find device "ceph-brx" 2023-07-08T12:38:45.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-08T12:38:45.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:45.922 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2023-07-08T12:38:45.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:45.982 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link delete ceph-brx 2023-07-08T12:38:46.052 INFO:teuthology.orchestra.run.smithi160.stderr:Cannot find device "ceph-brx" 2023-07-08T12:38:46.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-08T12:38:46.054 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-07-08T12:38:46.054 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-08T12:38:46.054 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-08T12:38:46.055 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-08T12:38:46.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.179+0000 7f9be7b76700 1 Processor -- start 2023-07-08T12:38:46.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.180+0000 7f9be7b76700 1 -- start start 2023-07-08T12:38:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.180+0000 7f9be7b76700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9be00cbb60 0x7f9be01588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.180+0000 7f9be7b76700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9be00ccd10 0x7f9be00cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.180+0000 7f9be7b76700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9be0069f30 0x7f9be006a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.180+0000 7f9be7b76700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be006d280 con 0x7f9be00cbb60 2023-07-08T12:38:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.180+0000 7f9be7b76700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be006cd50 con 0x7f9be00ccd10 2023-07-08T12:38:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.180+0000 7f9be7b76700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00c8ff0 con 0x7f9be0069f30 2023-07-08T12:38:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be5912700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9be00cbb60 0x7f9be01588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be5912700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9be00cbb60 0x7f9be01588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50308/0 (socket says 172.21.15.12:50308) 2023-07-08T12:38:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be5912700 1 -- 172.21.15.12:0/2946179197 learned_addr learned my addr 172.21.15.12:0/2946179197 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be5111700 1 --2- 172.21.15.12:0/2946179197 >> v2:172.21.15.160:3300/0 conn(0x7f9be00ccd10 0x7f9be00cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be6113700 1 --2- 172.21.15.12:0/2946179197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9be0069f30 0x7f9be006a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:46.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be5912700 1 -- 172.21.15.12:0/2946179197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9be0069f30 msgr2=0x7f9be006a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be5912700 1 --2- 172.21.15.12:0/2946179197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9be0069f30 0x7f9be006a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be5912700 1 -- 172.21.15.12:0/2946179197 >> v2:172.21.15.160:3300/0 conn(0x7f9be00ccd10 msgr2=0x7f9be00cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be5912700 1 --2- 172.21.15.12:0/2946179197 >> v2:172.21.15.160:3300/0 conn(0x7f9be00ccd10 0x7f9be00cb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be5912700 1 -- 172.21.15.12:0/2946179197 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be006d880 con 0x7f9be00cbb60 2023-07-08T12:38:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be6113700 1 --2- 172.21.15.12:0/2946179197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9be0069f30 0x7f9be006a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be5912700 1 --2- 172.21.15.12:0/2946179197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9be00cbb60 0x7f9be01588b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9be00c8da0 tx=0x7f9bdc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb6a56df9758710 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be4910700 1 -- 172.21.15.12:0/2946179197 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc00dd30 con 0x7f9be00cbb60 2023-07-08T12:38:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be4910700 1 -- 172.21.15.12:0/2946179197 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc00de90 con 0x7f9be00cbb60 2023-07-08T12:38:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.181+0000 7f9be4910700 1 -- 172.21.15.12:0/2946179197 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc010690 con 0x7f9be00cbb60 2023-07-08T12:38:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.182+0000 7f9be7b76700 1 -- 172.21.15.12:0/2946179197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9be00cbb60 msgr2=0x7f9be01588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.182+0000 7f9be7b76700 1 --2- 172.21.15.12:0/2946179197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9be00cbb60 0x7f9be01588b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9be00c8da0 tx=0x7f9bdc00be70 comp rx=0 tx=0).stop 2023-07-08T12:38:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.182+0000 7f9be7b76700 1 -- 172.21.15.12:0/2946179197 shutdown_connections 2023-07-08T12:38:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.182+0000 7f9be7b76700 1 --2- 172.21.15.12:0/2946179197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9be0069f30 0x7f9be006a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.182+0000 7f9be7b76700 1 --2- 172.21.15.12:0/2946179197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9be00cbb60 0x7f9be01588b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.182+0000 7f9be7b76700 1 --2- 172.21.15.12:0/2946179197 >> v2:172.21.15.160:3300/0 conn(0x7f9be00ccd10 0x7f9be00cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.182+0000 7f9be7b76700 1 -- 172.21.15.12:0/2946179197 >> 172.21.15.12:0/2946179197 conn(0x7f9be00b0390 msgr2=0x7f9be00b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.182+0000 7f9be7b76700 1 -- 172.21.15.12:0/2946179197 shutdown_connections 2023-07-08T12:38:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.182+0000 7f9be7b76700 1 -- 172.21.15.12:0/2946179197 wait complete. 2023-07-08T12:38:46.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.183+0000 7f9be7b76700 1 Processor -- start 2023-07-08T12:38:46.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.183+0000 7f9be7b76700 1 -- start start 2023-07-08T12:38:46.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be7b76700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9be0069f30 0x7f9be00c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:46.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be7b76700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9be00cbb60 0x7f9be00b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:46.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be7b76700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9be00ccd10 0x7f9be01f0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:46.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be7b76700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00b2ee0 con 0x7f9be00cbb60 2023-07-08T12:38:46.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be7b76700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be01f0850 con 0x7f9be0069f30 2023-07-08T12:38:46.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be7b76700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00b3180 con 0x7f9be00ccd10 2023-07-08T12:38:46.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be5111700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9be00cbb60 0x7f9be00b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:46.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be5111700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9be00cbb60 0x7f9be00b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50316/0 (socket says 172.21.15.12:50316) 2023-07-08T12:38:46.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be5111700 1 -- 172.21.15.12:0/175730150 learned_addr learned my addr 172.21.15.12:0/175730150 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:46.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be5912700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9be0069f30 0x7f9be00c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be6113700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9be00ccd10 0x7f9be01f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be5111700 1 -- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9be00ccd10 msgr2=0x7f9be01f0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be5111700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9be00ccd10 0x7f9be01f0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be5111700 1 -- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9be0069f30 msgr2=0x7f9be00c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be5111700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9be0069f30 0x7f9be00c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be5111700 1 -- 172.21.15.12:0/175730150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc00b7f0 con 0x7f9be00cbb60 2023-07-08T12:38:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.184+0000 7f9be5111700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9be00cbb60 0x7f9be00b29a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000e810 tx=0x7f9bd000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.185+0000 7f9bd67fc700 1 -- 172.21.15.12:0/175730150 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd000c070 con 0x7f9be00cbb60 2023-07-08T12:38:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.185+0000 7f9bd67fc700 1 -- 172.21.15.12:0/175730150 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd0013610 con 0x7f9be00cbb60 2023-07-08T12:38:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.185+0000 7f9bd67fc700 1 -- 172.21.15.12:0/175730150 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd00139c0 con 0x7f9be00cbb60 2023-07-08T12:38:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.185+0000 7f9be7b76700 1 -- 172.21.15.12:0/175730150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be00b3460 con 0x7f9be00cbb60 2023-07-08T12:38:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.185+0000 7f9be7b76700 1 -- 172.21.15.12:0/175730150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be00b3a20 con 0x7f9be00cbb60 2023-07-08T12:38:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.186+0000 7f9be7b76700 1 -- 172.21.15.12:0/175730150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be004ea60 con 0x7f9be00cbb60 2023-07-08T12:38:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.186+0000 7f9bd67fc700 1 -- 172.21.15.12:0/175730150 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9bd0013b20 con 0x7f9be00cbb60 2023-07-08T12:38:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.186+0000 7f9bd67fc700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9bc406ace0 0x7f9bc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:46.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.187+0000 7f9bd67fc700 1 -- 172.21.15.12:0/175730150 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9bd0089440 con 0x7f9be00cbb60 2023-07-08T12:38:46.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.187+0000 7f9be5912700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9bc406ace0 0x7f9bc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:46.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.187+0000 7f9be5912700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9bc406ace0 0x7f9bc406d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00beb0 tx=0x7f9bdc012000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:46.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.191+0000 7f9bd67fc700 1 -- 172.21.15.12:0/175730150 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9bd0057ca0 con 0x7f9be00cbb60 2023-07-08T12:38:46.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.435+0000 7f9be7b76700 1 -- 172.21.15.12:0/175730150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9be00b0e80 con 0x7f9be00cbb60 2023-07-08T12:38:46.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.435+0000 7f9bd67fc700 1 -- 172.21.15.12:0/175730150 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9bd0057830 con 0x7f9be00cbb60 2023-07-08T12:38:46.438 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-08T12:38:46.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.437+0000 7f9be7b76700 1 -- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9bc406ace0 msgr2=0x7f9bc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:46.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.437+0000 7f9be7b76700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9bc406ace0 0x7f9bc406d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00beb0 tx=0x7f9bdc012000 comp rx=0 tx=0).stop 2023-07-08T12:38:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.437+0000 7f9be7b76700 1 -- 172.21.15.12:0/175730150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9be00cbb60 msgr2=0x7f9be00b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.437+0000 7f9be7b76700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9be00cbb60 0x7f9be00b29a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000e810 tx=0x7f9bd000eb20 comp rx=0 tx=0).stop 2023-07-08T12:38:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.437+0000 7f9be7b76700 1 -- 172.21.15.12:0/175730150 shutdown_connections 2023-07-08T12:38:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.437+0000 7f9be7b76700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f9bc406ace0 0x7f9bc406d190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.437+0000 7f9be7b76700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9be00ccd10 0x7f9be01f0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.437+0000 7f9be7b76700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9be0069f30 0x7f9be00c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.437+0000 7f9be7b76700 1 --2- 172.21.15.12:0/175730150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9be00cbb60 0x7f9be00b29a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.437+0000 7f9be7b76700 1 -- 172.21.15.12:0/175730150 >> 172.21.15.12:0/175730150 conn(0x7f9be00b0390 msgr2=0x7f9be0157b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.437+0000 7f9be7b76700 1 -- 172.21.15.12:0/175730150 shutdown_connections 2023-07-08T12:38:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:46.437+0000 7f9be7b76700 1 -- 172.21.15.12:0/175730150 wait complete. 2023-07-08T12:38:46.451 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-08T12:38:46.452 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-08T12:38:46.452 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2023-07-08T12:38:46.452 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-08T12:38:46.453 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-08T12:38:46.453 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-08T12:38:46.453 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-08T12:38:46.453 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-08T12:38:46.454 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-08T12:38:46.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:46.454 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2023-07-08T12:38:46.471 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-08T12:38:46.472 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-08T12:38:46.472 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2023-07-08T12:38:46.472 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:46.472 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2023-07-08T12:38:46.472 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:46.472 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:46.473 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:46.473 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:46.473 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:46.473 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-08T12:38:46.473 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:46.473 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-08T12:38:46.473 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42123sec preferred_lft 42123sec 2023-07-08T12:38:46.473 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2023-07-08T12:38:46.474 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:46.474 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:46.474 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:46.474 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-08T12:38:46.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:46.475 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:46.475 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:46.475 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2023-07-08T12:38:46.476 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2023-07-08T12:38:46.476 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2023-07-08T12:38:46.476 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-08T12:38:46.476 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:46.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:46.590 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-08T12:38:46.618 INFO:teuthology.orchestra.run.smithi012.stdout:1 2023-07-08T12:38:46.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:46.621 DEBUG:teuthology.orchestra.run.smithi012:> route 2023-07-08T12:38:46.678 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2023-07-08T12:38:46.679 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-08T12:38:46.679 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-08T12:38:46.679 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-08T12:38:46.679 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-08T12:38:46.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:46.680 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:46.680 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:46.680 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-08T12:38:46.681 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-08T12:38:46.681 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-08T12:38:46.681 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:46.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:46.786 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2023-07-08T12:38:46.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:46.804 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2023-07-08T12:38:46.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:46.861 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:46.861 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:46.862 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-08T12:38:46.862 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-08T12:38:46.862 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:46.956 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-08T12:38:46.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:46.956 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:46.957 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:46.957 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-08T12:38:46.957 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-08T12:38:46.957 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-08T12:38:46.958 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-08T12:38:46.958 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-08T12:38:46.958 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:47.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:47.148 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:47.149 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:47.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2023-07-08T12:38:47.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2023-07-08T12:38:47.149 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:47.283 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-08T12:38:47.283 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-08T12:38:47.305 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-08T12:38:47.305 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-08T12:38:47.306 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-08T12:38:47.362 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-07-08T12:38:47.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:47.362 DEBUG:teuthology.orchestra.run.smithi012:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-08T12:38:47.450 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-08T12:38:47.450 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-07-08T12:38:47.450 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-07-08T12:38:47.588 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-08T12:38:47.588 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-07-08T12:38:47.589 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-07-08T12:38:47.589 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-07-08T12:38:47.589 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-08T12:38:47.661 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-08T12:38:47.661 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-07-08T12:38:47.662 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-07-08T12:38:47.662 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-07-08T12:38:47.662 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-08T12:38:47.662 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:6789,172.21.15.160:6789,172.21.15.160:6790" 2023-07-08T12:38:47.662 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.12:6789,172.21.15.160:6789,172.21.15.160:6790:/ 2023-07-08T12:38:47.662 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=71d270f5-58a7-4223-8d07-37436121729a" will pass to kernel 2023-07-08T12:38:47.664 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-08T12:38:47.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:47.665 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-08T12:38:47.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:47.694 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-08T12:38:47.765 INFO:teuthology.orchestra.run.smithi012.stdout:client4789 2023-07-08T12:38:47.765 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:38:47.766 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:38:47.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:47.823 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-08T12:38:47.896 INFO:teuthology.orchestra.run.smithi012.stdout:client4789 2023-07-08T12:38:47.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:47.897 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-08T12:38:47.969 INFO:teuthology.orchestra.run.smithi012.stdout:client4789 2023-07-08T12:38:47.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:47.970 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/71d270f5-58a7-4223-8d07-37436121729a.client4789/status') 2023-07-08T12:38:48.050 INFO:teuthology.orchestra.run.smithi012.stdout:instance: client.4789 (3)172.21.15.12:0/2339201284 2023-07-08T12:38:48.051 INFO:teuthology.orchestra.run.smithi012.stdout:blocklisted: false 2023-07-08T12:38:48.052 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records in 2023-07-08T12:38:48.052 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records out 2023-07-08T12:38:48.052 INFO:teuthology.orchestra.run.smithi012.stderr:70 bytes copied, 2.3575e-05 s, 3.0 MB/s 2023-07-08T12:38:48.053 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-07-08T12:38:48.053 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-08T12:38:48.053 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-08T12:38:48.054 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-08T12:38:48.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.215+0000 7f1e1372a700 1 Processor -- start 2023-07-08T12:38:48.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.216+0000 7f1e1372a700 1 -- start start 2023-07-08T12:38:48.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.216+0000 7f1e1372a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e0c1588b0 0x7f1e0c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:48.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.216+0000 7f1e1372a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1e0c0cad60 0x7f1e0c06a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:48.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.216+0000 7f1e1372a700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c06cd50 con 0x7f1e0c0cbb60 2023-07-08T12:38:48.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.216+0000 7f1e1372a700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e0c06d280 con 0x7f1e0c0cad60 2023-07-08T12:38:48.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.216+0000 7f1e1372a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0c8ff0 con 0x7f1e0c1588b0 2023-07-08T12:38:48.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.217+0000 7f1e11cc7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1e0c0cad60 0x7f1e0c06a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:48.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.217+0000 7f1e10cc5700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f1e0c0cbb60 0x7f1e0c070340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37722/0 (socket says 172.21.15.12:37722) 2023-07-08T12:38:48.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.217+0000 7f1e10cc5700 1 -- 172.21.15.12:0/595300816 learned_addr learned my addr 172.21.15.12:0/595300816 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:48.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.217+0000 7f1e114c6700 1 --2- 172.21.15.12:0/595300816 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e0c1588b0 0x7f1e0c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:48.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.217+0000 7f1e11cc7700 1 -- 172.21.15.12:0/595300816 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e0c1588b0 msgr2=0x7f1e0c069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:48.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.217+0000 7f1e11cc7700 1 --2- 172.21.15.12:0/595300816 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e0c1588b0 0x7f1e0c069f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:48.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.217+0000 7f1e11cc7700 1 -- 172.21.15.12:0/595300816 >> v1:172.21.15.12:6789/0 conn(0x7f1e0c0cbb60 legacy=0x7f1e0c070340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:48.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.217+0000 7f1e11cc7700 1 -- 172.21.15.12:0/595300816 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c06d880 con 0x7f1e0c0cad60 2023-07-08T12:38:48.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.217+0000 7f1e03fff700 1 -- 172.21.15.12:0/595300816 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846719923 0 0) 0x7f1e0c06cd50 con 0x7f1e0c0cbb60 2023-07-08T12:38:48.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.217+0000 7f1e11cc7700 1 --2- 172.21.15.12:0/595300816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e0c0cad60 0x7f1e0c06a470 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400aea0 tx=0x7f1e0400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2d16533778a0f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:48.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.217+0000 7f1e03fff700 1 -- 172.21.15.12:0/595300816 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400a2d0 con 0x7f1e0c0cad60 2023-07-08T12:38:48.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.217+0000 7f1e03fff700 1 -- 172.21.15.12:0/595300816 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0400c960 con 0x7f1e0c0cad60 2023-07-08T12:38:48.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.217+0000 7f1e03fff700 1 -- 172.21.15.12:0/595300816 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400cd10 con 0x7f1e0c0cad60 2023-07-08T12:38:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.218+0000 7f1e1372a700 1 -- 172.21.15.12:0/595300816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e0c0cad60 msgr2=0x7f1e0c06a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.218+0000 7f1e1372a700 1 --2- 172.21.15.12:0/595300816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e0c0cad60 0x7f1e0c06a470 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400aea0 tx=0x7f1e0400c330 comp rx=0 tx=0).stop 2023-07-08T12:38:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.218+0000 7f1e1372a700 1 -- 172.21.15.12:0/595300816 shutdown_connections 2023-07-08T12:38:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.218+0000 7f1e1372a700 1 --2- 172.21.15.12:0/595300816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e0c0cad60 0x7f1e0c06a470 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.218+0000 7f1e1372a700 1 --2- 172.21.15.12:0/595300816 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e0c1588b0 0x7f1e0c069f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.218+0000 7f1e1372a700 1 -- 172.21.15.12:0/595300816 >> 172.21.15.12:0/595300816 conn(0x7f1e0c0b0390 msgr2=0x7f1e0c0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.218+0000 7f1e1372a700 1 -- 172.21.15.12:0/595300816 shutdown_connections 2023-07-08T12:38:48.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.218+0000 7f1e1372a700 1 -- 172.21.15.12:0/595300816 wait complete. 2023-07-08T12:38:48.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.219+0000 7f1e1372a700 1 Processor -- start 2023-07-08T12:38:48.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.219+0000 7f1e1372a700 1 -- start start 2023-07-08T12:38:48.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e1372a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e0c0cad60 0x7f1e0c14d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:48.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e1372a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e0c0cbb60 0x7f1e0c0b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:48.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e1372a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e0c1588b0 0x7f1e0c0b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:48.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e1372a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c14de60 con 0x7f1e0c0cad60 2023-07-08T12:38:48.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e1372a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c14ecb0 con 0x7f1e0c1588b0 2023-07-08T12:38:48.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e1372a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c14e0a0 con 0x7f1e0c0cbb60 2023-07-08T12:38:48.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e114c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e0c0cad60 0x7f1e0c14d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:48.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e114c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e0c0cad60 0x7f1e0c14d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50332/0 (socket says 172.21.15.12:50332) 2023-07-08T12:38:48.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e114c6700 1 -- 172.21.15.12:0/163634590 learned_addr learned my addr 172.21.15.12:0/163634590 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:48.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e10cc5700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e0c0cbb60 0x7f1e0c0b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:48.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e11cc7700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e0c1588b0 0x7f1e0c0b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:48.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e114c6700 1 -- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e0c0cbb60 msgr2=0x7f1e0c0b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:48.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e114c6700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e0c0cbb60 0x7f1e0c0b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:48.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.220+0000 7f1e114c6700 1 -- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e0c1588b0 msgr2=0x7f1e0c0b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:48.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.221+0000 7f1e114c6700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e0c1588b0 0x7f1e0c0b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:48.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.221+0000 7f1e114c6700 1 -- 172.21.15.12:0/163634590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0400ab80 con 0x7f1e0c0cad60 2023-07-08T12:38:48.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.221+0000 7f1e114c6700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e0c0cad60 0x7f1e0c14d920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00d800 tx=0x7f1dfc00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:48.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.221+0000 7f1e01ffb700 1 -- 172.21.15.12:0/163634590 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0125f0 con 0x7f1e0c0cad60 2023-07-08T12:38:48.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.221+0000 7f1e01ffb700 1 -- 172.21.15.12:0/163634590 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc012750 con 0x7f1e0c0cad60 2023-07-08T12:38:48.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.221+0000 7f1e1372a700 1 -- 172.21.15.12:0/163634590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c14e380 con 0x7f1e0c0cad60 2023-07-08T12:38:48.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.221+0000 7f1e01ffb700 1 -- 172.21.15.12:0/163634590 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc012b00 con 0x7f1e0c0cad60 2023-07-08T12:38:48.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.221+0000 7f1e1372a700 1 -- 172.21.15.12:0/163634590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c14e940 con 0x7f1e0c0cad60 2023-07-08T12:38:48.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.222+0000 7f1e1372a700 1 -- 172.21.15.12:0/163634590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0c14ef90 con 0x7f1e0c0cad60 2023-07-08T12:38:48.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.223+0000 7f1e01ffb700 1 -- 172.21.15.12:0/163634590 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1dfc012c60 con 0x7f1e0c0cad60 2023-07-08T12:38:48.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.223+0000 7f1e01ffb700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1df006ace0 0x7f1df006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:48.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.223+0000 7f1e01ffb700 1 -- 172.21.15.12:0/163634590 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1dfc089720 con 0x7f1e0c0cad60 2023-07-08T12:38:48.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.223+0000 7f1e10cc5700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1df006ace0 0x7f1df006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:48.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.224+0000 7f1e10cc5700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1df006ace0 0x7f1df006d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0b3480 tx=0x7f1e08007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:48.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.227+0000 7f1e01ffb700 1 -- 172.21.15.12:0/163634590 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1dfc054570 con 0x7f1e0c0cad60 2023-07-08T12:38:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.477+0000 7f1e1372a700 1 -- 172.21.15.12:0/163634590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1e0c04ea60 con 0x7f1e0c0cad60 2023-07-08T12:38:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.478+0000 7f1e01ffb700 1 -- 172.21.15.12:0/163634590 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1dfc057b90 con 0x7f1e0c0cad60 2023-07-08T12:38:48.480 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-08T12:38:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.480+0000 7f1e1372a700 1 -- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1df006ace0 msgr2=0x7f1df006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:48.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.480+0000 7f1e1372a700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1df006ace0 0x7f1df006d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0b3480 tx=0x7f1e08007de0 comp rx=0 tx=0).stop 2023-07-08T12:38:48.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.480+0000 7f1e1372a700 1 -- 172.21.15.12:0/163634590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e0c0cad60 msgr2=0x7f1e0c14d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:48.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.480+0000 7f1e1372a700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e0c0cad60 0x7f1e0c14d920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00d800 tx=0x7f1dfc00db10 comp rx=0 tx=0).stop 2023-07-08T12:38:48.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.480+0000 7f1e1372a700 1 -- 172.21.15.12:0/163634590 shutdown_connections 2023-07-08T12:38:48.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.480+0000 7f1e1372a700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1df006ace0 0x7f1df006d190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:48.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.480+0000 7f1e1372a700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e0c1588b0 0x7f1e0c0b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.480+0000 7f1e1372a700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e0c0cad60 0x7f1e0c14d920 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.480+0000 7f1e1372a700 1 --2- 172.21.15.12:0/163634590 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e0c0cbb60 0x7f1e0c0b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.480+0000 7f1e1372a700 1 -- 172.21.15.12:0/163634590 >> 172.21.15.12:0/163634590 conn(0x7f1e0c0b0390 msgr2=0x7f1e0c0b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.480+0000 7f1e1372a700 1 -- 172.21.15.12:0/163634590 shutdown_connections 2023-07-08T12:38:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:48.480+0000 7f1e1372a700 1 -- 172.21.15.12:0/163634590 wait complete. 2023-07-08T12:38:48.494 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-08T12:38:48.495 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-08T12:38:48.495 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2023-07-08T12:38:48.495 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-07-08T12:38:48.495 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-08T12:38:48.495 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-08T12:38:48.496 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-08T12:38:48.496 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-08T12:38:48.496 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-07-08T12:38:48.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:48.496 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2023-07-08T12:38:48.514 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-08T12:38:48.515 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-08T12:38:48.515 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2023-07-08T12:38:48.515 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:48.515 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2023-07-08T12:38:48.515 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:48.515 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:48.515 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:48.516 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:48.516 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:48.516 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-08T12:38:48.516 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:48.516 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-08T12:38:48.516 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42121sec preferred_lft 42121sec 2023-07-08T12:38:48.516 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2023-07-08T12:38:48.517 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:48.517 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:48.517 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:48.517 INFO:teuthology.orchestra.run.smithi012.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-08T12:38:48.517 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether d6:f4:33:eb:94:9f brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:48.517 INFO:teuthology.orchestra.run.smithi012.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-08T12:38:48.517 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:48.517 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::84cf:ecff:fe20:a70e/64 scope link 2023-07-08T12:38:48.518 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:48.518 INFO:teuthology.orchestra.run.smithi012.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-08T12:38:48.518 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether d6:f4:33:eb:94:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-08T12:38:48.518 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::d4f4:33ff:feeb:949f/64 scope link tentative 2023-07-08T12:38:48.518 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:48.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:48.519 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-08T12:38:48.583 INFO:teuthology.orchestra.run.smithi012.stdout:1 2023-07-08T12:38:48.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:48.585 DEBUG:teuthology.orchestra.run.smithi012:> route 2023-07-08T12:38:48.644 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2023-07-08T12:38:48.644 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-08T12:38:48.644 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-08T12:38:48.644 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-08T12:38:48.645 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-08T12:38:48.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:48.645 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:48.646 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:48.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-08T12:38:48.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-08T12:38:48.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-08T12:38:48.646 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:48.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:48.752 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2023-07-08T12:38:48.769 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-08T12:38:48.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:48.771 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2023-07-08T12:38:48.827 INFO:teuthology.orchestra.run.smithi012.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-07-08T12:38:48.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:48.828 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:48.828 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:48.828 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-08T12:38:48.829 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-07-08T12:38:48.829 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:48.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:48.922 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-08T12:38:48.960 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-08T12:38:48.960 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-08T12:38:48.960 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2023-07-08T12:38:48.961 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:48.961 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2023-07-08T12:38:48.961 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:48.961 INFO:teuthology.orchestra.run.smithi012.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-08T12:38:48.961 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether de:3f:51:16:f2:c1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-08T12:38:48.961 INFO:teuthology.orchestra.run.smithi012.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-08T12:38:48.961 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:48.962 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::dc3f:51ff:fe16:f2c1/64 scope link 2023-07-08T12:38:48.962 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:48.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:48.973 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-08T12:38:49.055 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-08T12:38:49.056 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-08T12:38:49.056 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2023-07-08T12:38:49.056 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:49.056 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2023-07-08T12:38:49.056 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:49.056 INFO:teuthology.orchestra.run.smithi012.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-08T12:38:49.056 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether de:3f:51:16:f2:c1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-08T12:38:49.057 INFO:teuthology.orchestra.run.smithi012.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-08T12:38:49.057 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:49.057 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::dc3f:51ff:fe16:f2c1/64 scope link 2023-07-08T12:38:49.057 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:49.072 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-07-08T12:38:49.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:49.073 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:49.073 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:49.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-07-08T12:38:49.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-08T12:38:49.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-07-08T12:38:49.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-07-08T12:38:49.075 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-07-08T12:38:49.075 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:49.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:49.306 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:38:49.306 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:38:49.306 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.1 up 2023-07-08T12:38:49.307 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.1 master ceph-brx 2023-07-08T12:38:49.307 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:38:49.430 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-08T12:38:49.431 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-07-08T12:38:49.456 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-07-08T12:38:49.456 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-08T12:38:49.456 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-07-08T12:38:49.513 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-07-08T12:38:49.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:49.514 DEBUG:teuthology.orchestra.run.smithi012:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-07-08T12:38:49.599 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-07-08T12:38:49.600 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-07-08T12:38:49.600 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-07-08T12:38:49.608 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-07-08T12:38:49.608 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-07-08T12:38:49.609 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-07-08T12:38:49.609 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-07-08T12:38:49.609 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-08T12:38:49.637 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-07-08T12:38:49.637 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-07-08T12:38:49.638 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-07-08T12:38:49.638 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-07-08T12:38:49.638 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-08T12:38:49.638 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:6789,172.21.15.160:6789,172.21.15.160:6790" 2023-07-08T12:38:49.638 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.12:6789,172.21.15.160:6789,172.21.15.160:6790:/ 2023-07-08T12:38:49.638 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=71d270f5-58a7-4223-8d07-37436121729a" will pass to kernel 2023-07-08T12:38:49.642 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-08T12:38:49.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:49.643 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-07-08T12:38:49.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:49.683 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-08T12:38:49.756 INFO:teuthology.orchestra.run.smithi012.stdout:client4863 2023-07-08T12:38:49.756 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:38:49.757 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:38:49.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:49.813 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-08T12:38:49.885 INFO:teuthology.orchestra.run.smithi012.stdout:client4863 2023-07-08T12:38:49.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:49.886 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-08T12:38:49.958 INFO:teuthology.orchestra.run.smithi012.stdout:client4863 2023-07-08T12:38:49.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:49.959 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/71d270f5-58a7-4223-8d07-37436121729a.client4863/status') 2023-07-08T12:38:50.039 INFO:teuthology.orchestra.run.smithi012.stdout:instance: client.4863 (3)192.168.0.2:0/1433649713 2023-07-08T12:38:50.039 INFO:teuthology.orchestra.run.smithi012.stdout:blocklisted: false 2023-07-08T12:38:50.040 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records in 2023-07-08T12:38:50.040 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records out 2023-07-08T12:38:50.040 INFO:teuthology.orchestra.run.smithi012.stderr:69 bytes copied, 2.7006e-05 s, 2.6 MB/s 2023-07-08T12:38:50.041 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-07-08T12:38:50.041 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-08T12:38:50.041 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-08T12:38:50.041 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-08T12:38:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.204+0000 7f0189efa700 1 Processor -- start 2023-07-08T12:38:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.204+0000 7f0189efa700 1 -- start start 2023-07-08T12:38:50.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f0189efa700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f01840b1c70 0x7f01840ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:50.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f0189efa700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f018415d6c0 0x7f0184071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:50.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f0189efa700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f018415d230 con 0x7f01840ce1b0 2023-07-08T12:38:50.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f0189efa700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01840c9e60 con 0x7f01840b1c70 2023-07-08T12:38:50.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f0189efa700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01840ceac0 con 0x7f018415d6c0 2023-07-08T12:38:50.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f0182ffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f01840b1c70 0x7f01840ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:50.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f01837fe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f01840ce1b0 0x7f01840cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37740/0 (socket says 172.21.15.12:37740) 2023-07-08T12:38:50.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f01837fe700 1 -- 172.21.15.12:0/2998433724 learned_addr learned my addr 172.21.15.12:0/2998433724 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:50.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f0183fff700 1 --2- 172.21.15.12:0/2998433724 >> v2:172.21.15.160:3300/0 conn(0x7f018415d6c0 0x7f0184071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:50.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f0182ffd700 1 -- 172.21.15.12:0/2998433724 >> v2:172.21.15.160:3300/0 conn(0x7f018415d6c0 msgr2=0x7f0184071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:50.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f0182ffd700 1 --2- 172.21.15.12:0/2998433724 >> v2:172.21.15.160:3300/0 conn(0x7f018415d6c0 0x7f0184071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f0182ffd700 1 -- 172.21.15.12:0/2998433724 >> v1:172.21.15.12:6789/0 conn(0x7f01840ce1b0 legacy=0x7f01840cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f0182ffd700 1 -- 172.21.15.12:0/2998433724 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01840cdbc0 con 0x7f01840b1c70 2023-07-08T12:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f01827fc700 1 -- 172.21.15.12:0/2998433724 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114731857 0 0) 0x7f018415d230 con 0x7f01840ce1b0 2023-07-08T12:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f0182ffd700 1 --2- 172.21.15.12:0/2998433724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01840b1c70 0x7f01840ce580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f01840c6870 tx=0x7f0178009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e467bc171df508 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f01827fc700 1 -- 172.21.15.12:0/2998433724 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178011830 con 0x7f01840b1c70 2023-07-08T12:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f01827fc700 1 -- 172.21.15.12:0/2998433724 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0178011990 con 0x7f01840b1c70 2023-07-08T12:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.205+0000 7f01827fc700 1 -- 172.21.15.12:0/2998433724 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178011d20 con 0x7f01840b1c70 2023-07-08T12:38:50.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.206+0000 7f0189efa700 1 -- 172.21.15.12:0/2998433724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01840b1c70 msgr2=0x7f01840ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.206+0000 7f0189efa700 1 --2- 172.21.15.12:0/2998433724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01840b1c70 0x7f01840ce580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f01840c6870 tx=0x7f0178009b20 comp rx=0 tx=0).stop 2023-07-08T12:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.206+0000 7f0189efa700 1 -- 172.21.15.12:0/2998433724 shutdown_connections 2023-07-08T12:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.206+0000 7f0189efa700 1 --2- 172.21.15.12:0/2998433724 >> v2:172.21.15.160:3300/0 conn(0x7f018415d6c0 0x7f0184071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.206+0000 7f0189efa700 1 --2- 172.21.15.12:0/2998433724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01840b1c70 0x7f01840ce580 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.206+0000 7f0189efa700 1 -- 172.21.15.12:0/2998433724 >> 172.21.15.12:0/2998433724 conn(0x7f01840cc290 msgr2=0x7f01840cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.206+0000 7f0189efa700 1 -- 172.21.15.12:0/2998433724 shutdown_connections 2023-07-08T12:38:50.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.206+0000 7f0189efa700 1 -- 172.21.15.12:0/2998433724 wait complete. 2023-07-08T12:38:50.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.207+0000 7f0189efa700 1 Processor -- start 2023-07-08T12:38:50.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.207+0000 7f0189efa700 1 -- start start 2023-07-08T12:38:50.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.208+0000 7f0189efa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01840b1c70 0x7f0184159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:50.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.208+0000 7f0189efa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01840ce1b0 0x7f01841571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:50.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.208+0000 7f0189efa700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f018415d6c0 0x7f01840c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:50.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.208+0000 7f0189efa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0184159ae0 con 0x7f01840ce1b0 2023-07-08T12:38:50.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.208+0000 7f0189efa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0184158ec0 con 0x7f01840b1c70 2023-07-08T12:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.208+0000 7f0189efa700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0184159140 con 0x7f018415d6c0 2023-07-08T12:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.208+0000 7f0182ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01840ce1b0 0x7f01841571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.208+0000 7f0182ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01840ce1b0 0x7f01841571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50352/0 (socket says 172.21.15.12:50352) 2023-07-08T12:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.208+0000 7f0182ffd700 1 -- 172.21.15.12:0/1127426389 learned_addr learned my addr 172.21.15.12:0/1127426389 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.209+0000 7f01837fe700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01840b1c70 0x7f0184159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.209+0000 7f0183fff700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f018415d6c0 0x7f01840c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.209+0000 7f0182ffd700 1 -- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f018415d6c0 msgr2=0x7f01840c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:50.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.209+0000 7f0182ffd700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f018415d6c0 0x7f01840c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.209+0000 7f0182ffd700 1 -- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01840b1c70 msgr2=0x7f0184159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.209+0000 7f0182ffd700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01840b1c70 0x7f0184159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.209+0000 7f0182ffd700 1 -- 172.21.15.12:0/1127426389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01780094a0 con 0x7f01840ce1b0 2023-07-08T12:38:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.209+0000 7f0182ffd700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01840ce1b0 0x7f01841571e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0178009b60 tx=0x7f0178004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.209+0000 7f016bfff700 1 -- 172.21.15.12:0/1127426389 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01780038a0 con 0x7f01840ce1b0 2023-07-08T12:38:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.209+0000 7f016bfff700 1 -- 172.21.15.12:0/1127426389 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017801e070 con 0x7f01840ce1b0 2023-07-08T12:38:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.209+0000 7f0189efa700 1 -- 172.21.15.12:0/1127426389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0184159420 con 0x7f01840ce1b0 2023-07-08T12:38:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.209+0000 7f016bfff700 1 -- 172.21.15.12:0/1127426389 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178003c00 con 0x7f01840ce1b0 2023-07-08T12:38:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.209+0000 7f0189efa700 1 -- 172.21.15.12:0/1127426389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0184157720 con 0x7f01840ce1b0 2023-07-08T12:38:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.210+0000 7f0189efa700 1 -- 172.21.15.12:0/1127426389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0184159c20 con 0x7f01840ce1b0 2023-07-08T12:38:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.211+0000 7f016bfff700 1 -- 172.21.15.12:0/1127426389 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f017802d450 con 0x7f01840ce1b0 2023-07-08T12:38:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.211+0000 7f016bfff700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f017006ace0 0x7f017006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.211+0000 7f016bfff700 1 -- 172.21.15.12:0/1127426389 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f017808a840 con 0x7f01840ce1b0 2023-07-08T12:38:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.211+0000 7f01837fe700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f017006ace0 0x7f017006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:50.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.212+0000 7f01837fe700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f017006ace0 0x7f017006d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0174000f60 tx=0x7f0174007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:50.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.215+0000 7f016bfff700 1 -- 172.21.15.12:0/1127426389 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0184159c20 con 0x7f01840ce1b0 2023-07-08T12:38:50.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.461+0000 7f0189efa700 1 -- 172.21.15.12:0/1127426389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0184159c20 con 0x7f01840ce1b0 2023-07-08T12:38:50.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.461+0000 7f016bfff700 1 -- 172.21.15.12:0/1127426389 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0178020070 con 0x7f01840ce1b0 2023-07-08T12:38:50.463 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-08T12:38:50.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.463+0000 7f0189efa700 1 -- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f017006ace0 msgr2=0x7f017006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.463+0000 7f0189efa700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f017006ace0 0x7f017006d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0174000f60 tx=0x7f0174007c00 comp rx=0 tx=0).stop 2023-07-08T12:38:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.463+0000 7f0189efa700 1 -- 172.21.15.12:0/1127426389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01840ce1b0 msgr2=0x7f01841571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.463+0000 7f0189efa700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01840ce1b0 0x7f01841571e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0178009b60 tx=0x7f0178004750 comp rx=0 tx=0).stop 2023-07-08T12:38:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.463+0000 7f0189efa700 1 -- 172.21.15.12:0/1127426389 shutdown_connections 2023-07-08T12:38:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.463+0000 7f0189efa700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f017006ace0 0x7f017006d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.463+0000 7f0189efa700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f018415d6c0 0x7f01840c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:50.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.463+0000 7f0189efa700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01840b1c70 0x7f0184159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:50.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.463+0000 7f0189efa700 1 --2- 172.21.15.12:0/1127426389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01840ce1b0 0x7f01841571e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:50.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.463+0000 7f0189efa700 1 -- 172.21.15.12:0/1127426389 >> 172.21.15.12:0/1127426389 conn(0x7f01840cc290 msgr2=0x7f01840ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:50.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.463+0000 7f0189efa700 1 -- 172.21.15.12:0/1127426389 shutdown_connections 2023-07-08T12:38:50.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:50.463+0000 7f0189efa700 1 -- 172.21.15.12:0/1127426389 wait complete. 2023-07-08T12:38:50.477 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-08T12:38:50.478 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-08T12:38:50.478 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi160.front.sepia.ceph.com 2023-07-08T12:38:50.478 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-08T12:38:50.478 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-08T12:38:50.478 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-08T12:38:50.479 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-08T12:38:50.479 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-08T12:38:50.479 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-08T12:38:50.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:50.480 DEBUG:teuthology.orchestra.run.smithi160:> ip addr 2023-07-08T12:38:50.501 INFO:teuthology.orchestra.run.smithi160.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-08T12:38:50.501 INFO:teuthology.orchestra.run.smithi160.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-08T12:38:50.501 INFO:teuthology.orchestra.run.smithi160.stdout: inet 127.0.0.1/8 scope host lo 2023-07-08T12:38:50.502 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:50.502 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 ::1/128 scope host 2023-07-08T12:38:50.502 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:50.502 INFO:teuthology.orchestra.run.smithi160.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:50.502 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:80 brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:50.502 INFO:teuthology.orchestra.run.smithi160.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:50.502 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:81 brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:50.502 INFO:teuthology.orchestra.run.smithi160.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:50.503 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5a brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:50.503 INFO:teuthology.orchestra.run.smithi160.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-08T12:38:50.503 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5b brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:50.503 INFO:teuthology.orchestra.run.smithi160.stdout: inet 172.21.15.160/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-08T12:38:50.503 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft 42382sec preferred_lft 42382sec 2023-07-08T12:38:50.503 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::e9fa:c9f4:535e:371f/64 scope link noprefixroute 2023-07-08T12:38:50.503 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:50.504 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-08T12:38:50.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:50.504 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-07-08T12:38:50.505 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-07-08T12:38:50.505 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add name ceph-brx type bridge 2023-07-08T12:38:50.505 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip addr flush dev ceph-brx 2023-07-08T12:38:50.505 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set ceph-brx up 2023-07-08T12:38:50.506 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-08T12:38:50.506 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-07-08T12:38:50.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:50.673 DEBUG:teuthology.orchestra.run.smithi160:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-08T12:38:50.700 INFO:teuthology.orchestra.run.smithi160.stdout:1 2023-07-08T12:38:50.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:50.703 DEBUG:teuthology.orchestra.run.smithi160:> route 2023-07-08T12:38:50.774 INFO:teuthology.orchestra.run.smithi160.stdout:Kernel IP routing table 2023-07-08T12:38:50.775 INFO:teuthology.orchestra.run.smithi160.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-08T12:38:50.775 INFO:teuthology.orchestra.run.smithi160.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-08T12:38:50.775 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-08T12:38:50.775 INFO:teuthology.orchestra.run.smithi160.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-08T12:38:50.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:50.776 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-07-08T12:38:50.776 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-07-08T12:38:50.776 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-08T12:38:50.776 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-08T12:38:50.777 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-08T12:38:50.777 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-07-08T12:38:51.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:51.060 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2023-07-08T12:38:51.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:51.077 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list-id 2023-07-08T12:38:51.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:51.133 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-07-08T12:38:51.134 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-07-08T12:38:51.134 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-08T12:38:51.134 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-07-08T12:38:51.134 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-07-08T12:38:51.227 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-07-08T12:38:51.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:51.229 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-07-08T12:38:51.229 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-07-08T12:38:51.229 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-07-08T12:38:51.230 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-08T12:38:51.230 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-07-08T12:38:51.230 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-07-08T12:38:51.230 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-07-08T12:38:51.231 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-07-08T12:38:51.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:51.446 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-07-08T12:38:51.447 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-07-08T12:38:51.447 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set brx.0 up 2023-07-08T12:38:51.447 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set dev brx.0 master ceph-brx 2023-07-08T12:38:51.447 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-07-08T12:38:51.551 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-08T12:38:51.552 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-08T12:38:51.575 INFO:teuthology.orchestra.run.smithi160.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-08T12:38:51.575 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-08T12:38:51.576 DEBUG:teuthology.orchestra.run.smithi160:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-08T12:38:51.632 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-07-08T12:38:51.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:51.633 DEBUG:teuthology.orchestra.run.smithi160:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-07-08T12:38:51.749 INFO:teuthology.orchestra.run.smithi160.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-07-08T12:38:51.749 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-07-08T12:38:51.749 INFO:teuthology.orchestra.run.smithi160.stdout:invalid new device string format 2023-07-08T12:38:51.854 INFO:teuthology.orchestra.run.smithi160.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-07-08T12:38:51.854 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-07-08T12:38:51.855 INFO:teuthology.orchestra.run.smithi160.stdout:invalid new device string format 2023-07-08T12:38:51.855 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-07-08T12:38:51.855 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-08T12:38:51.882 INFO:teuthology.orchestra.run.smithi160.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-07-08T12:38:51.883 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-07-08T12:38:51.883 INFO:teuthology.orchestra.run.smithi160.stdout:invalid new device string format 2023-07-08T12:38:51.883 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-07-08T12:38:51.883 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-08T12:38:51.884 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: resolved to: "172.21.15.12:6789,172.21.15.160:6789,172.21.15.160:6790" 2023-07-08T12:38:51.884 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.12:6789,172.21.15.160:6789,172.21.15.160:6790:/ 2023-07-08T12:38:51.884 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=71d270f5-58a7-4223-8d07-37436121729a" will pass to kernel 2023-07-08T12:38:51.900 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-08T12:38:51.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:51.901 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-08T12:38:51.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:51.929 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-08T12:38:52.000 INFO:teuthology.orchestra.run.smithi160.stdout:client4634 2023-07-08T12:38:52.001 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:38:52.001 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:38:52.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:52.057 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-08T12:38:52.129 INFO:teuthology.orchestra.run.smithi160.stdout:client4634 2023-07-08T12:38:52.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:52.130 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-08T12:38:52.201 INFO:teuthology.orchestra.run.smithi160.stdout:client4634 2023-07-08T12:38:52.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:52.202 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/71d270f5-58a7-4223-8d07-37436121729a.client4634/status') 2023-07-08T12:38:52.282 INFO:teuthology.orchestra.run.smithi160.stdout:instance: client.4634 (3)192.168.0.1:0/3223137070 2023-07-08T12:38:52.282 INFO:teuthology.orchestra.run.smithi160.stdout:blocklisted: false 2023-07-08T12:38:52.283 INFO:teuthology.orchestra.run.smithi160.stderr:0+1 records in 2023-07-08T12:38:52.284 INFO:teuthology.orchestra.run.smithi160.stderr:0+1 records out 2023-07-08T12:38:52.284 INFO:teuthology.orchestra.run.smithi160.stderr:69 bytes copied, 2.3396e-05 s, 2.9 MB/s 2023-07-08T12:38:52.284 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-07-08T12:38:52.285 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-08T12:38:52.285 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-08T12:38:52.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-08T12:38:52.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.424+0000 7f60a1587700 1 Processor -- start 2023-07-08T12:38:52.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.424+0000 7f60a1587700 1 -- start start 2023-07-08T12:38:52.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f60a1587700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f609c159130 0x7f609c0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:52.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f60a1587700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f609c0ae730 0x7f609c0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:52.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f60a1587700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f609c0cd960 0x7f609c0ac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:52.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f60a1587700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609c0ae5a0 con 0x7f609c0cd960 2023-07-08T12:38:52.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f60a1587700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609c0af750 con 0x7f609c0ae730 2023-07-08T12:38:52.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f60a1587700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0cdda0 con 0x7f609c159130 2023-07-08T12:38:52.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f609b7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f609c0cd960 0x7f609c0ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:52.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f609b7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f609c0cd960 0x7f609c0ac480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50362/0 (socket says 172.21.15.12:50362) 2023-07-08T12:38:52.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f609b7fe700 1 -- 172.21.15.12:0/2576490693 learned_addr learned my addr 172.21.15.12:0/2576490693 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:52.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f609affd700 1 --2- 172.21.15.12:0/2576490693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f609c159130 0x7f609c0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:52.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f609a7fc700 1 --2- 172.21.15.12:0/2576490693 >> v2:172.21.15.160:3300/0 conn(0x7f609c0ae730 0x7f609c0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:52.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f609b7fe700 1 -- 172.21.15.12:0/2576490693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f609c159130 msgr2=0x7f609c0adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:52.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f609b7fe700 1 --2- 172.21.15.12:0/2576490693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f609c159130 0x7f609c0adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:52.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f609b7fe700 1 -- 172.21.15.12:0/2576490693 >> v2:172.21.15.160:3300/0 conn(0x7f609c0ae730 msgr2=0x7f609c0cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:52.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f609b7fe700 1 --2- 172.21.15.12:0/2576490693 >> v2:172.21.15.160:3300/0 conn(0x7f609c0ae730 0x7f609c0cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:52.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f609b7fe700 1 -- 172.21.15.12:0/2576490693 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c0d0760 con 0x7f609c0cd960 2023-07-08T12:38:52.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f609a7fc700 1 --2- 172.21.15.12:0/2576490693 >> v2:172.21.15.160:3300/0 conn(0x7f609c0ae730 0x7f609c0cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:52.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f609b7fe700 1 --2- 172.21.15.12:0/2576490693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f609c0cd960 0x7f609c0ac480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f609c06cee0 tx=0x7f609400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a88a13715fe5d6a server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:52.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f6099ffb700 1 -- 172.21.15.12:0/2576490693 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609400e070 con 0x7f609c0cd960 2023-07-08T12:38:52.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.425+0000 7f6099ffb700 1 -- 172.21.15.12:0/2576490693 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609400b910 con 0x7f609c0cd960 2023-07-08T12:38:52.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.426+0000 7f6099ffb700 1 -- 172.21.15.12:0/2576490693 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609400bcc0 con 0x7f609c0cd960 2023-07-08T12:38:52.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.426+0000 7f60a1587700 1 -- 172.21.15.12:0/2576490693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f609c0cd960 msgr2=0x7f609c0ac480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:52.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.426+0000 7f60a1587700 1 --2- 172.21.15.12:0/2576490693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f609c0cd960 0x7f609c0ac480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f609c06cee0 tx=0x7f609400b330 comp rx=0 tx=0).stop 2023-07-08T12:38:52.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.426+0000 7f60a1587700 1 -- 172.21.15.12:0/2576490693 shutdown_connections 2023-07-08T12:38:52.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.426+0000 7f60a1587700 1 --2- 172.21.15.12:0/2576490693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f609c159130 0x7f609c0adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:52.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.426+0000 7f60a1587700 1 --2- 172.21.15.12:0/2576490693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f609c0cd960 0x7f609c0ac480 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:52.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.426+0000 7f60a1587700 1 --2- 172.21.15.12:0/2576490693 >> v2:172.21.15.160:3300/0 conn(0x7f609c0ae730 0x7f609c0cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:52.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.426+0000 7f60a1587700 1 -- 172.21.15.12:0/2576490693 >> 172.21.15.12:0/2576490693 conn(0x7f609c098a30 msgr2=0x7f609c0cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:52.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.426+0000 7f60a1587700 1 -- 172.21.15.12:0/2576490693 shutdown_connections 2023-07-08T12:38:52.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.426+0000 7f60a1587700 1 -- 172.21.15.12:0/2576490693 wait complete. 2023-07-08T12:38:52.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.428+0000 7f60a1587700 1 Processor -- start 2023-07-08T12:38:52.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.428+0000 7f60a1587700 1 -- start start 2023-07-08T12:38:52.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f60a1587700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f609c0ae730 0x7f609c1f0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:52.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f60a1587700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f609c0cd960 0x7f609c1efae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:52.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f60a1587700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f609c159130 0x7f609c1e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:52.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f60a1587700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0b4480 con 0x7f609c0ae730 2023-07-08T12:38:52.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f60a1587700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c1f01e0 con 0x7f609c0cd960 2023-07-08T12:38:52.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f60a1587700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609c1f04c0 con 0x7f609c159130 2023-07-08T12:38:52.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f609affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f609c0ae730 0x7f609c1f0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:52.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f609affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f609c0ae730 0x7f609c1f0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50364/0 (socket says 172.21.15.12:50364) 2023-07-08T12:38:52.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f609affd700 1 -- 172.21.15.12:0/1895561509 learned_addr learned my addr 172.21.15.12:0/1895561509 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:52.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f609a7fc700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f609c0cd960 0x7f609c1efae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:52.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f609b7fe700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f609c159130 0x7f609c1e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:52.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f609affd700 1 -- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f609c159130 msgr2=0x7f609c1e50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:52.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f609affd700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f609c159130 0x7f609c1e50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f609affd700 1 -- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f609c0cd960 msgr2=0x7f609c1efae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f609affd700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f609c0cd960 0x7f609c1efae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f609affd700 1 -- 172.21.15.12:0/1895561509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6094009b30 con 0x7f609c0ae730 2023-07-08T12:38:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.429+0000 7f609b7fe700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f609c159130 0x7f609c1e50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.430+0000 7f609affd700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f609c0ae730 0x7f609c1f0f50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f608c00e9b0 tx=0x7f608c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.430+0000 7f60837fe700 1 -- 172.21.15.12:0/1895561509 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c009710 con 0x7f609c0ae730 2023-07-08T12:38:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.430+0000 7f60837fe700 1 -- 172.21.15.12:0/1895561509 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608c00c850 con 0x7f609c0ae730 2023-07-08T12:38:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.430+0000 7f60a1587700 1 -- 172.21.15.12:0/1895561509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609c1f07a0 con 0x7f609c0ae730 2023-07-08T12:38:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.430+0000 7f60837fe700 1 -- 172.21.15.12:0/1895561509 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c00cbc0 con 0x7f609c0ae730 2023-07-08T12:38:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.430+0000 7f60a1587700 1 -- 172.21.15.12:0/1895561509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609c1e5620 con 0x7f609c0ae730 2023-07-08T12:38:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.431+0000 7f60a1587700 1 -- 172.21.15.12:0/1895561509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609c1f0cc0 con 0x7f609c0ae730 2023-07-08T12:38:52.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.431+0000 7f60837fe700 1 -- 172.21.15.12:0/1895561509 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f608c00cd20 con 0x7f609c0ae730 2023-07-08T12:38:52.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.432+0000 7f60837fe700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f608806ace0 0x7f608806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:52.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.432+0000 7f60837fe700 1 -- 172.21.15.12:0/1895561509 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f608c0898c0 con 0x7f609c0ae730 2023-07-08T12:38:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.432+0000 7f609a7fc700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f608806ace0 0x7f608806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.432+0000 7f609a7fc700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f608806ace0 0x7f608806d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f60900098f0 tx=0x7f6090008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.436+0000 7f60837fe700 1 -- 172.21.15.12:0/1895561509 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f608c0580c0 con 0x7f609c0ae730 2023-07-08T12:38:52.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.687+0000 7f60a1587700 1 -- 172.21.15.12:0/1895561509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f609c04ea60 con 0x7f609c0ae730 2023-07-08T12:38:52.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.687+0000 7f60837fe700 1 -- 172.21.15.12:0/1895561509 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f608c057c50 con 0x7f609c0ae730 2023-07-08T12:38:52.690 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-08T12:38:52.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.690+0000 7f60a1587700 1 -- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f608806ace0 msgr2=0x7f608806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:52.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.690+0000 7f60a1587700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f608806ace0 0x7f608806d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f60900098f0 tx=0x7f6090008040 comp rx=0 tx=0).stop 2023-07-08T12:38:52.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.690+0000 7f60a1587700 1 -- 172.21.15.12:0/1895561509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f609c0ae730 msgr2=0x7f609c1f0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:52.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.690+0000 7f60a1587700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f609c0ae730 0x7f609c1f0f50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f608c00e9b0 tx=0x7f608c00ed70 comp rx=0 tx=0).stop 2023-07-08T12:38:52.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.690+0000 7f60a1587700 1 -- 172.21.15.12:0/1895561509 shutdown_connections 2023-07-08T12:38:52.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.690+0000 7f60a1587700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f608806ace0 0x7f608806d190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:52.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.690+0000 7f60a1587700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f609c159130 0x7f609c1e50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:52.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.690+0000 7f60a1587700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f609c0ae730 0x7f609c1f0f50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:52.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.690+0000 7f60a1587700 1 --2- 172.21.15.12:0/1895561509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f609c0cd960 0x7f609c1efae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:52.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.690+0000 7f60a1587700 1 -- 172.21.15.12:0/1895561509 >> 172.21.15.12:0/1895561509 conn(0x7f609c098a30 msgr2=0x7f609c0ad9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:52.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.690+0000 7f60a1587700 1 -- 172.21.15.12:0/1895561509 shutdown_connections 2023-07-08T12:38:52.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:52.690+0000 7f60a1587700 1 -- 172.21.15.12:0/1895561509 wait complete. 2023-07-08T12:38:52.705 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-08T12:38:52.705 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-08T12:38:52.706 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi160.front.sepia.ceph.com 2023-07-08T12:38:52.706 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-07-08T12:38:52.707 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-08T12:38:52.707 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-08T12:38:52.708 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-08T12:38:52.708 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-08T12:38:52.709 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-07-08T12:38:52.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:52.710 DEBUG:teuthology.orchestra.run.smithi160:> ip addr 2023-07-08T12:38:52.728 INFO:teuthology.orchestra.run.smithi160.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-08T12:38:52.728 INFO:teuthology.orchestra.run.smithi160.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-08T12:38:52.728 INFO:teuthology.orchestra.run.smithi160.stdout: inet 127.0.0.1/8 scope host lo 2023-07-08T12:38:52.728 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:52.728 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 ::1/128 scope host 2023-07-08T12:38:52.729 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:52.729 INFO:teuthology.orchestra.run.smithi160.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:52.729 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:80 brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:52.729 INFO:teuthology.orchestra.run.smithi160.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:52.729 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:81 brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:52.729 INFO:teuthology.orchestra.run.smithi160.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:38:52.729 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5a brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:52.729 INFO:teuthology.orchestra.run.smithi160.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-08T12:38:52.730 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5b brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:52.730 INFO:teuthology.orchestra.run.smithi160.stdout: inet 172.21.15.160/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-08T12:38:52.730 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft 42380sec preferred_lft 42380sec 2023-07-08T12:38:52.730 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::e9fa:c9f4:535e:371f/64 scope link noprefixroute 2023-07-08T12:38:52.730 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:52.730 INFO:teuthology.orchestra.run.smithi160.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-08T12:38:52.730 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 26:5a:e2:9d:6c:b6 brd ff:ff:ff:ff:ff:ff 2023-07-08T12:38:52.731 INFO:teuthology.orchestra.run.smithi160.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-08T12:38:52.731 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:52.731 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::f0c0:b5ff:fea7:9148/64 scope link 2023-07-08T12:38:52.731 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:52.731 INFO:teuthology.orchestra.run.smithi160.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-08T12:38:52.731 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 26:5a:e2:9d:6c:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-08T12:38:52.731 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::245a:e2ff:fe9d:6cb6/64 scope link tentative 2023-07-08T12:38:52.731 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:52.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:52.732 DEBUG:teuthology.orchestra.run.smithi160:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-08T12:38:52.793 INFO:teuthology.orchestra.run.smithi160.stdout:1 2023-07-08T12:38:52.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:52.795 DEBUG:teuthology.orchestra.run.smithi160:> route 2023-07-08T12:38:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:Kernel IP routing table 2023-07-08T12:38:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-08T12:38:52.854 INFO:teuthology.orchestra.run.smithi160.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-08T12:38:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-08T12:38:52.855 INFO:teuthology.orchestra.run.smithi160.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-08T12:38:52.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:52.856 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-07-08T12:38:52.857 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-07-08T12:38:52.857 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-08T12:38:52.858 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-08T12:38:52.858 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-08T12:38:52.859 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-07-08T12:38:52.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:52.959 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2023-07-08T12:38:52.975 INFO:teuthology.orchestra.run.smithi160.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-08T12:38:52.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:52.977 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list-id 2023-07-08T12:38:53.032 INFO:teuthology.orchestra.run.smithi160.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-07-08T12:38:53.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:53.033 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-07-08T12:38:53.033 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-07-08T12:38:53.033 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-08T12:38:53.034 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-07-08T12:38:53.034 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-07-08T12:38:53.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:53.126 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-08T12:38:53.169 INFO:teuthology.orchestra.run.smithi160.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-08T12:38:53.169 INFO:teuthology.orchestra.run.smithi160.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-08T12:38:53.170 INFO:teuthology.orchestra.run.smithi160.stdout: inet 127.0.0.1/8 scope host lo 2023-07-08T12:38:53.170 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:53.170 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 ::1/128 scope host 2023-07-08T12:38:53.170 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:53.170 INFO:teuthology.orchestra.run.smithi160.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-08T12:38:53.170 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether ee:54:3a:06:88:31 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-08T12:38:53.170 INFO:teuthology.orchestra.run.smithi160.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-08T12:38:53.171 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:53.171 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::ec54:3aff:fe06:8831/64 scope link tentative 2023-07-08T12:38:53.171 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:53.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:53.184 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-08T12:38:53.263 INFO:teuthology.orchestra.run.smithi160.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-08T12:38:53.263 INFO:teuthology.orchestra.run.smithi160.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-08T12:38:53.263 INFO:teuthology.orchestra.run.smithi160.stdout: inet 127.0.0.1/8 scope host lo 2023-07-08T12:38:53.263 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:53.263 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 ::1/128 scope host 2023-07-08T12:38:53.264 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:53.264 INFO:teuthology.orchestra.run.smithi160.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-08T12:38:53.264 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether ee:54:3a:06:88:31 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-08T12:38:53.264 INFO:teuthology.orchestra.run.smithi160.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-08T12:38:53.264 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:53.264 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::ec54:3aff:fe06:8831/64 scope link tentative 2023-07-08T12:38:53.264 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:38:53.276 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-07-08T12:38:53.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:53.276 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-07-08T12:38:53.277 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-07-08T12:38:53.277 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-07-08T12:38:53.277 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-08T12:38:53.277 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-07-08T12:38:53.278 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-07-08T12:38:53.278 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-07-08T12:38:53.278 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-07-08T12:38:53.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:53.512 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-07-08T12:38:53.512 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-07-08T12:38:53.512 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set brx.1 up 2023-07-08T12:38:53.512 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set dev brx.1 master ceph-brx 2023-07-08T12:38:53.513 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-07-08T12:38:53.624 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-08T12:38:53.625 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-07-08T12:38:53.646 INFO:teuthology.orchestra.run.smithi160.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-07-08T12:38:53.647 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-08T12:38:53.647 DEBUG:teuthology.orchestra.run.smithi160:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-07-08T12:38:53.703 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-07-08T12:38:53.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:53.703 DEBUG:teuthology.orchestra.run.smithi160:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-07-08T12:38:53.790 INFO:teuthology.orchestra.run.smithi160.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-07-08T12:38:53.790 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-07-08T12:38:53.790 INFO:teuthology.orchestra.run.smithi160.stdout:invalid new device string format 2023-07-08T12:38:53.798 INFO:teuthology.orchestra.run.smithi160.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-07-08T12:38:53.798 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-07-08T12:38:53.798 INFO:teuthology.orchestra.run.smithi160.stdout:invalid new device string format 2023-07-08T12:38:53.798 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-07-08T12:38:53.798 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-08T12:38:53.872 INFO:teuthology.orchestra.run.smithi160.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-07-08T12:38:53.872 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-07-08T12:38:53.872 INFO:teuthology.orchestra.run.smithi160.stdout:invalid new device string format 2023-07-08T12:38:53.873 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-07-08T12:38:53.873 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-08T12:38:53.873 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: resolved to: "172.21.15.12:6789,172.21.15.160:6789,172.21.15.160:6790" 2023-07-08T12:38:53.873 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.12:6789,172.21.15.160:6789,172.21.15.160:6790:/ 2023-07-08T12:38:53.873 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=71d270f5-58a7-4223-8d07-37436121729a" will pass to kernel 2023-07-08T12:38:53.875 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-08T12:38:53.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:53.876 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-07-08T12:38:53.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:53.904 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-08T12:38:53.976 INFO:teuthology.orchestra.run.smithi160.stdout:client4881 2023-07-08T12:38:53.977 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:38:53.977 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:38:54.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:54.033 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-08T12:38:54.105 INFO:teuthology.orchestra.run.smithi160.stdout:client4881 2023-07-08T12:38:54.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:54.106 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-08T12:38:54.177 INFO:teuthology.orchestra.run.smithi160.stdout:client4881 2023-07-08T12:38:54.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:38:54.179 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/71d270f5-58a7-4223-8d07-37436121729a.client4881/status') 2023-07-08T12:38:54.257 INFO:teuthology.orchestra.run.smithi160.stdout:instance: client.4881 (3)172.21.15.160:0/4183412563 2023-07-08T12:38:54.258 INFO:teuthology.orchestra.run.smithi160.stdout:blocklisted: false 2023-07-08T12:38:54.259 INFO:teuthology.orchestra.run.smithi160.stderr:0+1 records in 2023-07-08T12:38:54.259 INFO:teuthology.orchestra.run.smithi160.stderr:0+1 records out 2023-07-08T12:38:54.259 INFO:teuthology.orchestra.run.smithi160.stderr:71 bytes copied, 4.0968e-05 s, 1.7 MB/s 2023-07-08T12:38:54.259 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-07-08T12:38:54.382 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:54.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.520+0000 7fdb94261700 1 Processor -- start 2023-07-08T12:38:54.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.520+0000 7fdb94261700 1 -- start start 2023-07-08T12:38:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.521+0000 7fdb94261700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdb8c0cd7a0 0x7fdb8c0cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.521+0000 7fdb94261700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb8c0cd1c0 0x7fdb8c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.521+0000 7fdb94261700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8c1594b0 con 0x7fdb8c0caf90 2023-07-08T12:38:54.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.521+0000 7fdb94261700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8c0ce0b0 con 0x7fdb8c0cd7a0 2023-07-08T12:38:54.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.521+0000 7fdb94261700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0ca1d0 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.521+0000 7fdb917fc700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdb8c0caf90 0x7fdb8c0ccab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37770/0 (socket says 172.21.15.12:37770) 2023-07-08T12:38:54.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.521+0000 7fdb917fc700 1 -- 172.21.15.12:0/2462690319 learned_addr learned my addr 172.21.15.12:0/2462690319 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:54.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.522+0000 7fdb91ffd700 1 --2- 172.21.15.12:0/2462690319 >> v2:172.21.15.160:3300/0 conn(0x7fdb8c0cd7a0 0x7fdb8c0cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:54.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.522+0000 7fdb927fe700 1 --2- 172.21.15.12:0/2462690319 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb8c0cd1c0 0x7fdb8c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:54.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.522+0000 7fdb90ffb700 1 -- 172.21.15.12:0/2462690319 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732357090 0 0) 0x7fdb8c1594b0 con 0x7fdb8c0caf90 2023-07-08T12:38:54.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.522+0000 7fdb90ffb700 1 -- 172.21.15.12:0/2462690319 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb780035b0 con 0x7fdb8c0caf90 2023-07-08T12:38:54.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.522+0000 7fdb91ffd700 1 -- 172.21.15.12:0/2462690319 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb8c0cd1c0 msgr2=0x7fdb8c069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:54.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.522+0000 7fdb91ffd700 1 --2- 172.21.15.12:0/2462690319 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb8c0cd1c0 0x7fdb8c069f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.522+0000 7fdb91ffd700 1 -- 172.21.15.12:0/2462690319 >> v1:172.21.15.12:6789/0 conn(0x7fdb8c0caf90 legacy=0x7fdb8c0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:54.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.522+0000 7fdb91ffd700 1 -- 172.21.15.12:0/2462690319 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c0ca4b0 con 0x7fdb8c0cd7a0 2023-07-08T12:38:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.522+0000 7fdb927fe700 1 --2- 172.21.15.12:0/2462690319 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb8c0cd1c0 0x7fdb8c069f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:38:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.522+0000 7fdb91ffd700 1 --2- 172.21.15.12:0/2462690319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb8c0cd7a0 0x7fdb8c0cdb70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009840 tx=0x7fdb7c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e293edf231895880 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.522+0000 7fdb90ffb700 1 -- 172.21.15.12:0/2462690319 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c011830 con 0x7fdb8c0cd7a0 2023-07-08T12:38:54.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.522+0000 7fdb90ffb700 1 -- 172.21.15.12:0/2462690319 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c011990 con 0x7fdb8c0cd7a0 2023-07-08T12:38:54.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.522+0000 7fdb90ffb700 1 -- 172.21.15.12:0/2462690319 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c011d20 con 0x7fdb8c0cd7a0 2023-07-08T12:38:54.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.523+0000 7fdb94261700 1 -- 172.21.15.12:0/2462690319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb8c0cd7a0 msgr2=0x7fdb8c0cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:54.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.523+0000 7fdb94261700 1 --2- 172.21.15.12:0/2462690319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb8c0cd7a0 0x7fdb8c0cdb70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009840 tx=0x7fdb7c009b50 comp rx=0 tx=0).stop 2023-07-08T12:38:54.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.523+0000 7fdb94261700 1 -- 172.21.15.12:0/2462690319 shutdown_connections 2023-07-08T12:38:54.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.523+0000 7fdb94261700 1 --2- 172.21.15.12:0/2462690319 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb8c0cd1c0 0x7fdb8c069f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.523+0000 7fdb94261700 1 --2- 172.21.15.12:0/2462690319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb8c0cd7a0 0x7fdb8c0cdb70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.523+0000 7fdb94261700 1 -- 172.21.15.12:0/2462690319 >> 172.21.15.12:0/2462690319 conn(0x7fdb8c0cb9f0 msgr2=0x7fdb8c0cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:54.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.523+0000 7fdb94261700 1 -- 172.21.15.12:0/2462690319 shutdown_connections 2023-07-08T12:38:54.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.523+0000 7fdb94261700 1 -- 172.21.15.12:0/2462690319 wait complete. 2023-07-08T12:38:54.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.524+0000 7fdb94261700 1 Processor -- start 2023-07-08T12:38:54.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.524+0000 7fdb94261700 1 -- start start 2023-07-08T12:38:54.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb94261700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb8c0caf90 0x7fdb8c0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:54.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb94261700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c0cd1c0 0x7fdb8c1551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:54.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb94261700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb8c0cd7a0 0x7fdb8c0ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:54.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb94261700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0c6e50 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb94261700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0c2a10 con 0x7fdb8c0caf90 2023-07-08T12:38:54.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb94261700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0c5ba0 con 0x7fdb8c0cd7a0 2023-07-08T12:38:54.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb917fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c0cd1c0 0x7fdb8c1551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:54.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb917fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c0cd1c0 0x7fdb8c1551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50380/0 (socket says 172.21.15.12:50380) 2023-07-08T12:38:54.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb917fc700 1 -- 172.21.15.12:0/952137380 learned_addr learned my addr 172.21.15.12:0/952137380 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:54.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb927fe700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb8c0cd7a0 0x7fdb8c0ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:54.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb91ffd700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb8c0caf90 0x7fdb8c0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:54.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb917fc700 1 -- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb8c0cd7a0 msgr2=0x7fdb8c0ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:54.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb917fc700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb8c0cd7a0 0x7fdb8c0ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.525+0000 7fdb917fc700 1 -- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb8c0caf90 msgr2=0x7fdb8c0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.526+0000 7fdb917fc700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb8c0caf90 0x7fdb8c0b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.526+0000 7fdb917fc700 1 -- 172.21.15.12:0/952137380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb84009a70 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.526+0000 7fdb917fc700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c0cd1c0 0x7fdb8c1551b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c700 tx=0x7fdb8800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.526+0000 7fdb827fc700 1 -- 172.21.15.12:0/952137380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88005d70 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.526+0000 7fdb827fc700 1 -- 172.21.15.12:0/952137380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb880070d0 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.526+0000 7fdb94261700 1 -- 172.21.15.12:0/952137380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7c0094a0 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.526+0000 7fdb827fc700 1 -- 172.21.15.12:0/952137380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88007460 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.526+0000 7fdb94261700 1 -- 172.21.15.12:0/952137380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c1f2c20 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.527+0000 7fdb94261700 1 -- 172.21.15.12:0/952137380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8c06b890 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.527+0000 7fdb827fc700 1 -- 172.21.15.12:0/952137380 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdb880075c0 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.528+0000 7fdb827fc700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdb7006ace0 0x7fdb7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:54.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.528+0000 7fdb827fc700 1 -- 172.21.15.12:0/952137380 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdb88088c40 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.528+0000 7fdb91ffd700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdb7006ace0 0x7fdb7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:54.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.528+0000 7fdb91ffd700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdb7006ace0 0x7fdb7006d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c000c00 tx=0x7fdb7c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:54.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.532+0000 7fdb827fc700 1 -- 172.21.15.12:0/952137380 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdb880539c0 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.734+0000 7fdb94261700 1 -- 172.21.15.12:0/952137380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb8c066e80 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.735+0000 7fdb827fc700 1 -- 172.21.15.12:0/952137380 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20203 (secure 0 0 0) 0x7fdb88056fe0 con 0x7fdb8c0cd1c0 2023-07-08T12:38:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:49.386462+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:54.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.737+0000 7fdb94261700 1 -- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdb7006ace0 msgr2=0x7fdb7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:54.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.737+0000 7fdb94261700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdb7006ace0 0x7fdb7006d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c000c00 tx=0x7fdb7c01c040 comp rx=0 tx=0).stop 2023-07-08T12:38:54.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.737+0000 7fdb94261700 1 -- 172.21.15.12:0/952137380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c0cd1c0 msgr2=0x7fdb8c1551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.737+0000 7fdb94261700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c0cd1c0 0x7fdb8c1551b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c700 tx=0x7fdb8800ca10 comp rx=0 tx=0).stop 2023-07-08T12:38:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.738+0000 7fdb94261700 1 -- 172.21.15.12:0/952137380 shutdown_connections 2023-07-08T12:38:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.738+0000 7fdb94261700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdb7006ace0 0x7fdb7006d190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.738+0000 7fdb94261700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb8c0cd7a0 0x7fdb8c0ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.738+0000 7fdb94261700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb8c0caf90 0x7fdb8c0b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.738+0000 7fdb94261700 1 --2- 172.21.15.12:0/952137380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c0cd1c0 0x7fdb8c1551b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.738+0000 7fdb94261700 1 -- 172.21.15.12:0/952137380 >> 172.21.15.12:0/952137380 conn(0x7fdb8c0cb9f0 msgr2=0x7fdb8c1586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:54.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.738+0000 7fdb94261700 1 -- 172.21.15.12:0/952137380 shutdown_connections 2023-07-08T12:38:54.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.738+0000 7fdb94261700 1 -- 172.21.15.12:0/952137380 wait complete. 2023-07-08T12:38:54.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-08T12:38:05.660194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-08T12:38:06.920177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-08T12:38:10.592749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-08T12:38:54.756 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:38:54.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.882+0000 7fa237457700 1 Processor -- start 2023-07-08T12:38:54.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.882+0000 7fa237457700 1 -- start start 2023-07-08T12:38:54.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa237457700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2300b4590 0x7fa2300b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:54.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa237457700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa23006c210 0x7fa23015c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:54.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa237457700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2300b4ea0 con 0x7fa23006bda0 2023-07-08T12:38:54.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa237457700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23006b390 con 0x7fa23006c210 2023-07-08T12:38:54.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa237457700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23015d3d0 con 0x7fa2300b4590 2023-07-08T12:38:54.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa2359f4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa23006c210 0x7fa23015c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa2359f4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa23006c210 0x7fa23015c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50390/0 (socket says 172.21.15.12:50390) 2023-07-08T12:38:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa2359f4700 1 -- 172.21.15.12:0/2801797005 learned_addr learned my addr 172.21.15.12:0/2801797005 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa2351f3700 1 --2- 172.21.15.12:0/2801797005 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2300b4590 0x7fa2300b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa2359f4700 1 -- 172.21.15.12:0/2801797005 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2300b4590 msgr2=0x7fa2300b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa2359f4700 1 --2- 172.21.15.12:0/2801797005 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2300b4590 0x7fa2300b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa2359f4700 1 -- 172.21.15.12:0/2801797005 >> v1:172.21.15.160:6789/0 conn(0x7fa23006bda0 legacy=0x7fa2300d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa2359f4700 1 -- 172.21.15.12:0/2801797005 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23015d6b0 con 0x7fa23006c210 2023-07-08T12:38:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa2359f4700 1 --2- 172.21.15.12:0/2801797005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa23006c210 0x7fa23015c590 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa228009e50 tx=0x7fa22800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf8f5bf7bba7363 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa227fff700 1 -- 172.21.15.12:0/2801797005 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22800e070 con 0x7fa23006c210 2023-07-08T12:38:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa227fff700 1 -- 172.21.15.12:0/2801797005 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22800b910 con 0x7fa23006c210 2023-07-08T12:38:54.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.883+0000 7fa227fff700 1 -- 172.21.15.12:0/2801797005 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22800bcc0 con 0x7fa23006c210 2023-07-08T12:38:54.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.884+0000 7fa237457700 1 -- 172.21.15.12:0/2801797005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa23006c210 msgr2=0x7fa23015c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:54.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.884+0000 7fa237457700 1 --2- 172.21.15.12:0/2801797005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa23006c210 0x7fa23015c590 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa228009e50 tx=0x7fa22800b330 comp rx=0 tx=0).stop 2023-07-08T12:38:54.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.884+0000 7fa237457700 1 -- 172.21.15.12:0/2801797005 shutdown_connections 2023-07-08T12:38:54.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.884+0000 7fa237457700 1 --2- 172.21.15.12:0/2801797005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa23006c210 0x7fa23015c590 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.884+0000 7fa237457700 1 --2- 172.21.15.12:0/2801797005 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2300b4590 0x7fa2300b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.884+0000 7fa237457700 1 -- 172.21.15.12:0/2801797005 >> 172.21.15.12:0/2801797005 conn(0x7fa2300cf590 msgr2=0x7fa2300cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:54.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.884+0000 7fa237457700 1 -- 172.21.15.12:0/2801797005 shutdown_connections 2023-07-08T12:38:54.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.884+0000 7fa237457700 1 -- 172.21.15.12:0/2801797005 wait complete. 2023-07-08T12:38:54.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.885+0000 7fa237457700 1 Processor -- start 2023-07-08T12:38:54.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.885+0000 7fa237457700 1 -- start start 2023-07-08T12:38:54.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.886+0000 7fa237457700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa23006bda0 0x7fa230154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:54.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.886+0000 7fa237457700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa23006c210 0x7fa2300c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:54.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.886+0000 7fa237457700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2300b4590 0x7fa2301e4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:54.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.886+0000 7fa237457700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2301e37a0 con 0x7fa23006c210 2023-07-08T12:38:54.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.886+0000 7fa237457700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa230155270 con 0x7fa2300b4590 2023-07-08T12:38:54.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.886+0000 7fa237457700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2300cc6f0 con 0x7fa23006bda0 2023-07-08T12:38:54.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.886+0000 7fa2349f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa23006c210 0x7fa2300c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:54.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.886+0000 7fa2349f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa23006c210 0x7fa2300c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50394/0 (socket says 172.21.15.12:50394) 2023-07-08T12:38:54.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.886+0000 7fa2349f2700 1 -- 172.21.15.12:0/2373941006 learned_addr learned my addr 172.21.15.12:0/2373941006 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:54.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.886+0000 7fa2351f3700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa23006bda0 0x7fa230154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:54.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.887+0000 7fa2359f4700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2300b4590 0x7fa2301e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:54.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.887+0000 7fa2349f2700 1 -- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa23006bda0 msgr2=0x7fa230154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:54.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.887+0000 7fa2349f2700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa23006bda0 0x7fa230154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.887+0000 7fa2349f2700 1 -- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2300b4590 msgr2=0x7fa2301e4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:54.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.887+0000 7fa2349f2700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2300b4590 0x7fa2301e4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:54.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.887+0000 7fa2349f2700 1 -- 172.21.15.12:0/2373941006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa228009b30 con 0x7fa23006c210 2023-07-08T12:38:54.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.887+0000 7fa2349f2700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa23006c210 0x7fa2300c8ec0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa22000c4e0 tx=0x7fa22000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:54.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.887+0000 7fa225ffb700 1 -- 172.21.15.12:0/2373941006 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2200052d0 con 0x7fa23006c210 2023-07-08T12:38:54.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.887+0000 7fa225ffb700 1 -- 172.21.15.12:0/2373941006 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa220005430 con 0x7fa23006c210 2023-07-08T12:38:54.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.887+0000 7fa225ffb700 1 -- 172.21.15.12:0/2373941006 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2200038c0 con 0x7fa23006c210 2023-07-08T12:38:54.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.887+0000 7fa237457700 1 -- 172.21.15.12:0/2373941006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2300cc9d0 con 0x7fa23006c210 2023-07-08T12:38:54.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.887+0000 7fa237457700 1 -- 172.21.15.12:0/2373941006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2301e3430 con 0x7fa23006c210 2023-07-08T12:38:54.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.888+0000 7fa237457700 1 -- 172.21.15.12:0/2373941006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2301e35c0 con 0x7fa23006c210 2023-07-08T12:38:54.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.888+0000 7fa225ffb700 1 -- 172.21.15.12:0/2373941006 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa220003a20 con 0x7fa23006c210 2023-07-08T12:38:54.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.889+0000 7fa225ffb700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa21406ace0 0x7fa21406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:54.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.889+0000 7fa225ffb700 1 -- 172.21.15.12:0/2373941006 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa220088c00 con 0x7fa23006c210 2023-07-08T12:38:54.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.889+0000 7fa2351f3700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa21406ace0 0x7fa21406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:54.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.889+0000 7fa2351f3700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa21406ace0 0x7fa21406d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa2301f5ad0 tx=0x7fa22c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:54.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:54.893+0000 7fa225ffb700 1 -- 172.21.15.12:0/2373941006 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa22008f080 con 0x7fa23006c210 2023-07-08T12:38:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.091+0000 7fa237457700 1 -- 172.21.15.12:0/2373941006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2300cf1d0 con 0x7fa23006c210 2023-07-08T12:38:55.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.092+0000 7fa225ffb700 1 -- 172.21.15.12:0/2373941006 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa220017020 con 0x7fa23006c210 2023-07-08T12:38:55.094 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:38:55.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.094+0000 7fa237457700 1 -- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa21406ace0 msgr2=0x7fa21406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.094+0000 7fa237457700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa21406ace0 0x7fa21406d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa2301f5ad0 tx=0x7fa22c008040 comp rx=0 tx=0).stop 2023-07-08T12:38:55.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.094+0000 7fa237457700 1 -- 172.21.15.12:0/2373941006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa23006c210 msgr2=0x7fa2300c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.094+0000 7fa237457700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa23006c210 0x7fa2300c8ec0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa22000c4e0 tx=0x7fa22000c8a0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.094+0000 7fa237457700 1 -- 172.21.15.12:0/2373941006 shutdown_connections 2023-07-08T12:38:55.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.094+0000 7fa237457700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa21406ace0 0x7fa21406d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.094+0000 7fa237457700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2300b4590 0x7fa2301e4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.094+0000 7fa237457700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa23006bda0 0x7fa230154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.094+0000 7fa237457700 1 --2- 172.21.15.12:0/2373941006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa23006c210 0x7fa2300c8ec0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.094+0000 7fa237457700 1 -- 172.21.15.12:0/2373941006 >> 172.21.15.12:0/2373941006 conn(0x7fa2300cf590 msgr2=0x7fa2300b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:55.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.094+0000 7fa237457700 1 -- 172.21.15.12:0/2373941006 shutdown_connections 2023-07-08T12:38:55.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.094+0000 7fa237457700 1 -- 172.21.15.12:0/2373941006 wait complete. 2023-07-08T12:38:55.108 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:38:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.234+0000 7f894b1e7700 1 Processor -- start 2023-07-08T12:38:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.234+0000 7f894b1e7700 1 -- start start 2023-07-08T12:38:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.235+0000 7f894b1e7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f89440aeec0 0x7f894406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.235+0000 7f894b1e7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f89440cdb80 0x7f89440cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.235+0000 7f894b1e7700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894415d770 con 0x7f894406ebe0 2023-07-08T12:38:55.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.235+0000 7f894b1e7700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894415a600 con 0x7f89440aeec0 2023-07-08T12:38:55.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.235+0000 7f894b1e7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894415cb60 con 0x7f89440cdb80 2023-07-08T12:38:55.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.235+0000 7f8948f83700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f89440aeec0 0x7f894406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.235+0000 7f8948f83700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f89440aeec0 0x7f894406cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50400/0 (socket says 172.21.15.12:50400) 2023-07-08T12:38:55.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.235+0000 7f8948f83700 1 -- 172.21.15.12:0/2822659381 learned_addr learned my addr 172.21.15.12:0/2822659381 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:55.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.235+0000 7f8949784700 1 --2- 172.21.15.12:0/2822659381 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f89440cdb80 0x7f89440cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.236+0000 7f8948f83700 1 -- 172.21.15.12:0/2822659381 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f89440cdb80 msgr2=0x7f89440cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.236+0000 7f8948f83700 1 --2- 172.21.15.12:0/2822659381 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f89440cdb80 0x7f89440cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.236+0000 7f8948f83700 1 -- 172.21.15.12:0/2822659381 >> v1:172.21.15.160:6789/0 conn(0x7f894406ebe0 legacy=0x7f89440cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.236+0000 7f8948f83700 1 -- 172.21.15.12:0/2822659381 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894415ce40 con 0x7f89440aeec0 2023-07-08T12:38:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.236+0000 7f8948f83700 1 --2- 172.21.15.12:0/2822659381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89440aeec0 0x7f894406cff0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8938006fd0 tx=0x7f893800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17abc3813cb221de server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.236+0000 7f89437fe700 1 -- 172.21.15.12:0/2822659381 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938005e30 con 0x7f89440aeec0 2023-07-08T12:38:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.236+0000 7f89437fe700 1 -- 172.21.15.12:0/2822659381 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8938004030 con 0x7f89440aeec0 2023-07-08T12:38:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.236+0000 7f89437fe700 1 -- 172.21.15.12:0/2822659381 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89380043e0 con 0x7f89440aeec0 2023-07-08T12:38:55.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.236+0000 7f894b1e7700 1 -- 172.21.15.12:0/2822659381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89440aeec0 msgr2=0x7f894406cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.236+0000 7f894b1e7700 1 --2- 172.21.15.12:0/2822659381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89440aeec0 0x7f894406cff0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8938006fd0 tx=0x7f893800be70 comp rx=0 tx=0).stop 2023-07-08T12:38:55.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.237+0000 7f894b1e7700 1 -- 172.21.15.12:0/2822659381 shutdown_connections 2023-07-08T12:38:55.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.237+0000 7f894b1e7700 1 --2- 172.21.15.12:0/2822659381 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f89440cdb80 0x7f89440cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.237+0000 7f894b1e7700 1 --2- 172.21.15.12:0/2822659381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89440aeec0 0x7f894406cff0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.237+0000 7f894b1e7700 1 -- 172.21.15.12:0/2822659381 >> 172.21.15.12:0/2822659381 conn(0x7f8944098b50 msgr2=0x7f894406b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:55.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.237+0000 7f894b1e7700 1 -- 172.21.15.12:0/2822659381 shutdown_connections 2023-07-08T12:38:55.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.237+0000 7f894b1e7700 1 -- 172.21.15.12:0/2822659381 wait complete. 2023-07-08T12:38:55.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.238+0000 7f894b1e7700 1 Processor -- start 2023-07-08T12:38:55.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.238+0000 7f894b1e7700 1 -- start start 2023-07-08T12:38:55.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f894b1e7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f894406ebe0 0x7f8944157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f894b1e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89440aeec0 0x7f89440c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f894b1e7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89440cdb80 0x7f89440b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f894b1e7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89440b05c0 con 0x7f89440aeec0 2023-07-08T12:38:55.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f894b1e7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89440b08a0 con 0x7f89440cdb80 2023-07-08T12:38:55.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f894b1e7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89440b0b80 con 0x7f894406ebe0 2023-07-08T12:38:55.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f8943fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89440aeec0 0x7f89440c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f8943fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89440aeec0 0x7f89440c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50408/0 (socket says 172.21.15.12:50408) 2023-07-08T12:38:55.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f8943fff700 1 -- 172.21.15.12:0/3684393698 learned_addr learned my addr 172.21.15.12:0/3684393698 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:55.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f8948f83700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f894406ebe0 0x7f8944157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f8949784700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89440cdb80 0x7f89440b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f8943fff700 1 -- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f894406ebe0 msgr2=0x7f8944157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f8943fff700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f894406ebe0 0x7f8944157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f8943fff700 1 -- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89440cdb80 msgr2=0x7f89440b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f8943fff700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89440cdb80 0x7f89440b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f8943fff700 1 -- 172.21.15.12:0/3684393698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893800b7a0 con 0x7f89440aeec0 2023-07-08T12:38:55.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f8949784700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89440cdb80 0x7f89440b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:55.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.239+0000 7f8943fff700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89440aeec0 0x7f89440c3ca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f893000c4e0 tx=0x7f893000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:55.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.240+0000 7f89417fa700 1 -- 172.21.15.12:0/3684393698 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300052d0 con 0x7f89440aeec0 2023-07-08T12:38:55.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.240+0000 7f89417fa700 1 -- 172.21.15.12:0/3684393698 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8930005430 con 0x7f89440aeec0 2023-07-08T12:38:55.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.240+0000 7f894b1e7700 1 -- 172.21.15.12:0/3684393698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89440b0e60 con 0x7f89440aeec0 2023-07-08T12:38:55.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.240+0000 7f89417fa700 1 -- 172.21.15.12:0/3684393698 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300038c0 con 0x7f89440aeec0 2023-07-08T12:38:55.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.240+0000 7f894b1e7700 1 -- 172.21.15.12:0/3684393698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89441579a0 con 0x7f89440aeec0 2023-07-08T12:38:55.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.241+0000 7f894b1e7700 1 -- 172.21.15.12:0/3684393698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89440b1380 con 0x7f89440aeec0 2023-07-08T12:38:55.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.241+0000 7f89417fa700 1 -- 172.21.15.12:0/3684393698 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8930003a20 con 0x7f89440aeec0 2023-07-08T12:38:55.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.241+0000 7f89417fa700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f892806ace0 0x7f892806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.241+0000 7f89417fa700 1 -- 172.21.15.12:0/3684393698 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8930088cf0 con 0x7f89440aeec0 2023-07-08T12:38:55.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.242+0000 7f8948f83700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f892806ace0 0x7f892806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.242+0000 7f8948f83700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f892806ace0 0x7f892806d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f893800b340 tx=0x7f893801b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:55.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.246+0000 7f89417fa700 1 -- 172.21.15.12:0/3684393698 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f89440b1380 con 0x7f89440aeec0 2023-07-08T12:38:55.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.443+0000 7f894b1e7700 1 -- 172.21.15.12:0/3684393698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f89440b1380 con 0x7f89440aeec0 2023-07-08T12:38:55.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.443+0000 7f89417fa700 1 -- 172.21.15.12:0/3684393698 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8930017020 con 0x7f89440aeec0 2023-07-08T12:38:55.446 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:38:55.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.445+0000 7f894b1e7700 1 -- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f892806ace0 msgr2=0x7f892806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.445+0000 7f894b1e7700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f892806ace0 0x7f892806d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f893800b340 tx=0x7f893801b040 comp rx=0 tx=0).stop 2023-07-08T12:38:55.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.445+0000 7f894b1e7700 1 -- 172.21.15.12:0/3684393698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89440aeec0 msgr2=0x7f89440c3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.445+0000 7f894b1e7700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89440aeec0 0x7f89440c3ca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f893000c4e0 tx=0x7f893000c7f0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.445+0000 7f894b1e7700 1 -- 172.21.15.12:0/3684393698 shutdown_connections 2023-07-08T12:38:55.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.445+0000 7f894b1e7700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f892806ace0 0x7f892806d190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.445+0000 7f894b1e7700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89440cdb80 0x7f89440b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.445+0000 7f894b1e7700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f894406ebe0 0x7f8944157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.445+0000 7f894b1e7700 1 --2- 172.21.15.12:0/3684393698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89440aeec0 0x7f89440c3ca0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.445+0000 7f894b1e7700 1 -- 172.21.15.12:0/3684393698 >> 172.21.15.12:0/3684393698 conn(0x7f8944098b50 msgr2=0x7f894406d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:55.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.446+0000 7f894b1e7700 1 -- 172.21.15.12:0/3684393698 shutdown_connections 2023-07-08T12:38:55.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.446+0000 7f894b1e7700 1 -- 172.21.15.12:0/3684393698 wait complete. 2023-07-08T12:38:55.460 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-08T12:38:55.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.587+0000 7f20a7b5a700 1 Processor -- start 2023-07-08T12:38:55.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.587+0000 7f20a7b5a700 1 -- start start 2023-07-08T12:38:55.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a7b5a700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f20a015a6d0 0x7f20a015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a7b5a700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a0158130 con 0x7f20a0161ce0 2023-07-08T12:38:55.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a7b5a700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a015b450 con 0x7f20a00ce560 2023-07-08T12:38:55.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a7b5a700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a015b730 con 0x7f20a015a6d0 2023-07-08T12:38:55.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a60f7700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f20a0161ce0 0x7f20a01640c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37814/0 (socket says 172.21.15.12:37814) 2023-07-08T12:38:55.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a60f7700 1 -- 172.21.15.12:0/3726526992 learned_addr learned my addr 172.21.15.12:0/3726526992 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:55.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a50f5700 1 --2- 172.21.15.12:0/3726526992 >> v2:172.21.15.160:3300/0 conn(0x7f20a015a6d0 0x7f20a015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422482399 0 0) 0x7f20a0158130 con 0x7f20a0161ce0 2023-07-08T12:38:55.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f208c0035b0 con 0x7f20a0161ce0 2023-07-08T12:38:55.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 80022168 0 0) 0x7f208c0035b0 con 0x7f20a0161ce0 2023-07-08T12:38:55.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 >> v2:172.21.15.160:3300/0 conn(0x7f20a015a6d0 msgr2=0x7f20a015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:55.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 --2- 172.21.15.12:0/3726526992 >> v2:172.21.15.160:3300/0 conn(0x7f20a015a6d0 0x7f20a015ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 >> v1:172.21.15.160:6789/0 conn(0x7f20a00ce560 legacy=0x7f20a00ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a006e850 con 0x7f20a0161ce0 2023-07-08T12:38:55.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f20980019f0 con 0x7f20a0161ce0 2023-07-08T12:38:55.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 >> v1:172.21.15.12:6789/0 conn(0x7f20a0161ce0 legacy=0x7f20a01640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 --2- 172.21.15.12:0/3726526992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c003a00 0x7f208c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 --2- 172.21.15.12:0/3726526992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f208c004850 0x7f208c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 --2- 172.21.15.12:0/3726526992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f208c005200 0x7f208c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a015b730 con 0x7f208c005200 2023-07-08T12:38:55.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c00c080 con 0x7f208c004850 2023-07-08T12:38:55.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.588+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208c00c240 con 0x7f208c003a00 2023-07-08T12:38:55.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.589+0000 7f20a60f7700 1 --2- 172.21.15.12:0/3726526992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f208c005200 0x7f208c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.589+0000 7f20a50f5700 1 --2- 172.21.15.12:0/3726526992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f208c004850 0x7f208c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.589+0000 7f20a58f6700 1 --2- 172.21.15.12:0/3726526992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c003a00 0x7f208c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.589+0000 7f20a60f7700 1 -- 172.21.15.12:0/3726526992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c003a00 msgr2=0x7f208c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:55.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.589+0000 7f20a60f7700 1 --2- 172.21.15.12:0/3726526992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c003a00 0x7f208c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.589+0000 7f20a60f7700 1 -- 172.21.15.12:0/3726526992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f208c004850 msgr2=0x7f208c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:55.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.589+0000 7f20a60f7700 1 --2- 172.21.15.12:0/3726526992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f208c004850 0x7f208c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.589+0000 7f20a60f7700 1 -- 172.21.15.12:0/3726526992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c0035b0 con 0x7f208c005200 2023-07-08T12:38:55.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.589+0000 7f20a60f7700 1 -- 172.21.15.12:0/3726526992 reap_dead start 2023-07-08T12:38:55.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.589+0000 7f20a60f7700 1 --2- 172.21.15.12:0/3726526992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f208c005200 0x7f208c00ba10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f209800a800 tx=0x7f208c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef97c6ea6cad472f server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:55.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.589+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209800a2f0 con 0x7f208c005200 2023-07-08T12:38:55.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.589+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2098012620 con 0x7f208c005200 2023-07-08T12:38:55.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.589+0000 7f20a48f4700 1 -- 172.21.15.12:0/3726526992 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20980129d0 con 0x7f208c005200 2023-07-08T12:38:55.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.590+0000 7f20a7b5a700 1 -- 172.21.15.12:0/3726526992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f208c005200 msgr2=0x7f208c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.590+0000 7f20a7b5a700 1 --2- 172.21.15.12:0/3726526992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f208c005200 0x7f208c00ba10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f209800a800 tx=0x7f208c005880 comp rx=0 tx=0).stop 2023-07-08T12:38:55.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.590+0000 7f20a7b5a700 1 -- 172.21.15.12:0/3726526992 shutdown_connections 2023-07-08T12:38:55.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.590+0000 7f20a7b5a700 1 --2- 172.21.15.12:0/3726526992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f208c005200 0x7f208c00ba10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.590+0000 7f20a7b5a700 1 -- 172.21.15.12:0/3726526992 >> 172.21.15.12:0/3726526992 conn(0x7f20a0099c60 msgr2=0x7f20a0089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:55.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.590+0000 7f20a7b5a700 1 -- 172.21.15.12:0/3726526992 shutdown_connections 2023-07-08T12:38:55.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.590+0000 7f20a7b5a700 1 -- 172.21.15.12:0/3726526992 wait complete. 2023-07-08T12:38:55.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.591+0000 7f20a7b5a700 1 Processor -- start 2023-07-08T12:38:55.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.591+0000 7f20a7b5a700 1 -- start start 2023-07-08T12:38:55.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.592+0000 7f20a7b5a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f208c005200 0x7f20a01ef570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.592+0000 7f20a7b5a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00b4180 0x7f20a00b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.592+0000 7f20a7b5a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a0152760 0x7f20a0152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.592+0000 7f20a7b5a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00b4b30 con 0x7f208c005200 2023-07-08T12:38:55.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.592+0000 7f20a7b5a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0153110 con 0x7f20a0152760 2023-07-08T12:38:55.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.592+0000 7f20a7b5a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00b07f0 con 0x7f20a00b4180 2023-07-08T12:38:55.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.592+0000 7f20a58f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f208c005200 0x7f20a01ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a58f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f208c005200 0x7f20a01ef570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50426/0 (socket says 172.21.15.12:50426) 2023-07-08T12:38:55.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a58f6700 1 -- 172.21.15.12:0/411956180 learned_addr learned my addr 172.21.15.12:0/411956180 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:55.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a60f7700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a0152760 0x7f20a0152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a50f5700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00b4180 0x7f20a00b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a58f6700 1 -- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00b4180 msgr2=0x7f20a00b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a58f6700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00b4180 0x7f20a00b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a58f6700 1 -- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a0152760 msgr2=0x7f20a0152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a58f6700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a0152760 0x7f20a0152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a58f6700 1 -- 172.21.15.12:0/411956180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00c7320 con 0x7f208c005200 2023-07-08T12:38:55.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a60f7700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a0152760 0x7f20a0152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:55.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a50f5700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00b4180 0x7f20a00b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:55.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a58f6700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f208c005200 0x7f20a01ef570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f209c00f7f0 tx=0x7f209c00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:55.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20927fc700 1 -- 172.21.15.12:0/411956180 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c005320 con 0x7f208c005200 2023-07-08T12:38:55.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20927fc700 1 -- 172.21.15.12:0/411956180 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c005480 con 0x7f208c005200 2023-07-08T12:38:55.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a7b5a700 1 -- 172.21.15.12:0/411956180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a00b4d40 con 0x7f208c005200 2023-07-08T12:38:55.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20927fc700 1 -- 172.21.15.12:0/411956180 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c006aa0 con 0x7f208c005200 2023-07-08T12:38:55.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.593+0000 7f20a7b5a700 1 -- 172.21.15.12:0/411956180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a00b5300 con 0x7f208c005200 2023-07-08T12:38:55.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.594+0000 7f20a7b5a700 1 -- 172.21.15.12:0/411956180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a00b0b20 con 0x7f208c005200 2023-07-08T12:38:55.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.595+0000 7f20927fc700 1 -- 172.21.15.12:0/411956180 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f209c006c00 con 0x7f208c005200 2023-07-08T12:38:55.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.595+0000 7f20927fc700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f208406ace0 0x7f208406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.595+0000 7f20927fc700 1 -- 172.21.15.12:0/411956180 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f209c0890d0 con 0x7f208c005200 2023-07-08T12:38:55.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.595+0000 7f20a50f5700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f208406ace0 0x7f208406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.596+0000 7f20a50f5700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f208406ace0 0x7f208406d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2094009470 tx=0x7f2094008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:55.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.599+0000 7f20927fc700 1 -- 172.21.15.12:0/411956180 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f209c053e50 con 0x7f208c005200 2023-07-08T12:38:55.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.796+0000 7f20a7b5a700 1 -- 172.21.15.12:0/411956180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20a004ea60 con 0x7f208c005200 2023-07-08T12:38:55.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.796+0000 7f20927fc700 1 -- 172.21.15.12:0/411956180 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f209c057470 con 0x7f208c005200 2023-07-08T12:38:55.799 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:38:55.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.798+0000 7f20a7b5a700 1 -- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f208406ace0 msgr2=0x7f208406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.798+0000 7f20a7b5a700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f208406ace0 0x7f208406d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2094009470 tx=0x7f2094008040 comp rx=0 tx=0).stop 2023-07-08T12:38:55.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.798+0000 7f20a7b5a700 1 -- 172.21.15.12:0/411956180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f208c005200 msgr2=0x7f20a01ef570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.798+0000 7f20a7b5a700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f208c005200 0x7f20a01ef570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f209c00f7f0 tx=0x7f209c00fbb0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.798+0000 7f20a7b5a700 1 -- 172.21.15.12:0/411956180 shutdown_connections 2023-07-08T12:38:55.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.798+0000 7f20a7b5a700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f208406ace0 0x7f208406d190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.798+0000 7f20a7b5a700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a0152760 0x7f20a0152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.798+0000 7f20a7b5a700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f208c005200 0x7f20a01ef570 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.798+0000 7f20a7b5a700 1 --2- 172.21.15.12:0/411956180 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00b4180 0x7f20a00b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.798+0000 7f20a7b5a700 1 -- 172.21.15.12:0/411956180 >> 172.21.15.12:0/411956180 conn(0x7f20a0099c60 msgr2=0x7f20a00729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:55.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.798+0000 7f20a7b5a700 1 -- 172.21.15.12:0/411956180 shutdown_connections 2023-07-08T12:38:55.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.798+0000 7f20a7b5a700 1 -- 172.21.15.12:0/411956180 wait complete. 2023-07-08T12:38:55.812 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-08T12:38:55.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.937+0000 7fa543eb6700 1 Processor -- start 2023-07-08T12:38:55.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.938+0000 7fa543eb6700 1 -- start start 2023-07-08T12:38:55.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.938+0000 7fa543eb6700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c15a6d0 0x7fa53c15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.938+0000 7fa543eb6700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa53c161ce0 0x7fa53c1640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.938+0000 7fa543eb6700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa53c158130 con 0x7fa53c0ce560 2023-07-08T12:38:55.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.938+0000 7fa543eb6700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53c15b730 con 0x7fa53c161ce0 2023-07-08T12:38:55.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.938+0000 7fa543eb6700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c15b450 con 0x7fa53c15a6d0 2023-07-08T12:38:55.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa541c52700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa53c0ce560 0x7fa53c0ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37830/0 (socket says 172.21.15.12:37830) 2023-07-08T12:38:55.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa541c52700 1 -- 172.21.15.12:0/909436963 learned_addr learned my addr 172.21.15.12:0/909436963 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:55.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa541451700 1 --2- 172.21.15.12:0/909436963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c15a6d0 0x7fa53c15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168929799 0 0) 0x7fa53c158130 con 0x7fa53c0ce560 2023-07-08T12:38:55.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa542453700 1 --2- 172.21.15.12:0/909436963 >> v2:172.21.15.160:3300/0 conn(0x7fa53c161ce0 0x7fa53c1640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5280035b0 con 0x7fa53c0ce560 2023-07-08T12:38:55.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 437296592 0 0) 0x7fa5280035b0 con 0x7fa53c0ce560 2023-07-08T12:38:55.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c15a6d0 msgr2=0x7fa53c15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:55.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 --2- 172.21.15.12:0/909436963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c15a6d0 0x7fa53c15ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 >> v2:172.21.15.160:3300/0 conn(0x7fa53c161ce0 msgr2=0x7fa53c1640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:55.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 --2- 172.21.15.12:0/909436963 >> v2:172.21.15.160:3300/0 conn(0x7fa53c161ce0 0x7fa53c1640c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c06e850 con 0x7fa53c0ce560 2023-07-08T12:38:55.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7fa52c001670 con 0x7fa53c0ce560 2023-07-08T12:38:55.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 >> v1:172.21.15.12:6789/0 conn(0x7fa53c0ce560 legacy=0x7fa53c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 --2- 172.21.15.12:0/909436963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528003a00 0x7fa528003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 --2- 172.21.15.12:0/909436963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa528004430 0x7fa5280048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 --2- 172.21.15.12:0/909436963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c15a6d0 0x7fa528004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c15b730 con 0x7fa528004430 2023-07-08T12:38:55.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c15b450 con 0x7fa528003a00 2023-07-08T12:38:55.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52800ba10 con 0x7fa53c15a6d0 2023-07-08T12:38:55.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa541451700 1 --2- 172.21.15.12:0/909436963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa528004430 0x7fa5280048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa541c52700 1 --2- 172.21.15.12:0/909436963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528003a00 0x7fa528003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.939+0000 7fa542453700 1 --2- 172.21.15.12:0/909436963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c15a6d0 0x7fa528004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.940+0000 7fa541451700 1 -- 172.21.15.12:0/909436963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c15a6d0 msgr2=0x7fa528004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:55.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.940+0000 7fa541451700 1 --2- 172.21.15.12:0/909436963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c15a6d0 0x7fa528004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.940+0000 7fa541451700 1 -- 172.21.15.12:0/909436963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528003a00 msgr2=0x7fa528003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:55.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.940+0000 7fa541451700 1 --2- 172.21.15.12:0/909436963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa528003a00 0x7fa528003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.940+0000 7fa541451700 1 -- 172.21.15.12:0/909436963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5280035b0 con 0x7fa528004430 2023-07-08T12:38:55.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.940+0000 7fa541451700 1 --2- 172.21.15.12:0/909436963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa528004430 0x7fa5280048f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa538009b60 tx=0x7fa528005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444d5f97564cca7 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:55.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.940+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53800e6c0 con 0x7fa528004430 2023-07-08T12:38:55.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.940+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53800e820 con 0x7fa528004430 2023-07-08T12:38:55.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.940+0000 7fa540c50700 1 -- 172.21.15.12:0/909436963 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53800eb90 con 0x7fa528004430 2023-07-08T12:38:55.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.940+0000 7fa543eb6700 1 -- 172.21.15.12:0/909436963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa528004430 msgr2=0x7fa5280048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.940+0000 7fa543eb6700 1 --2- 172.21.15.12:0/909436963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa528004430 0x7fa5280048f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa538009b60 tx=0x7fa528005880 comp rx=0 tx=0).stop 2023-07-08T12:38:55.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.941+0000 7fa542453700 1 -- 172.21.15.12:0/909436963 reap_dead start 2023-07-08T12:38:55.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.941+0000 7fa543eb6700 1 -- 172.21.15.12:0/909436963 shutdown_connections 2023-07-08T12:38:55.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.941+0000 7fa543eb6700 1 -- 172.21.15.12:0/909436963 >> 172.21.15.12:0/909436963 conn(0x7fa53c099c60 msgr2=0x7fa53c089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:55.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.941+0000 7fa543eb6700 1 -- 172.21.15.12:0/909436963 shutdown_connections 2023-07-08T12:38:55.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.941+0000 7fa543eb6700 1 -- 172.21.15.12:0/909436963 wait complete. 2023-07-08T12:38:55.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.942+0000 7fa543eb6700 1 Processor -- start 2023-07-08T12:38:55.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.942+0000 7fa543eb6700 1 -- start start 2023-07-08T12:38:55.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa543eb6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa528004430 0x7fa53c14e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa543eb6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa53c0c5d10 0x7fa53c1f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa543eb6700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c1f4000 0x7fa53c0c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa543eb6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c0ccb50 con 0x7fa528004430 2023-07-08T12:38:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa543eb6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c0cbf40 con 0x7fa53c0c5d10 2023-07-08T12:38:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa543eb6700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c0cc220 con 0x7fa53c1f4000 2023-07-08T12:38:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa541c52700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa528004430 0x7fa53c14e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa541c52700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa528004430 0x7fa53c14e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50446/0 (socket says 172.21.15.12:50446) 2023-07-08T12:38:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa541c52700 1 -- 172.21.15.12:0/4149962487 learned_addr learned my addr 172.21.15.12:0/4149962487 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa541451700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa53c0c5d10 0x7fa53c1f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa542453700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c1f4000 0x7fa53c0c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa541c52700 1 -- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c1f4000 msgr2=0x7fa53c0c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa541c52700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c1f4000 0x7fa53c0c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa541c52700 1 -- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa53c0c5d10 msgr2=0x7fa53c1f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:55.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa541c52700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa53c0c5d10 0x7fa53c1f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:55.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa541c52700 1 -- 172.21.15.12:0/4149962487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c0cc500 con 0x7fa528004430 2023-07-08T12:38:55.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa541451700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa53c0c5d10 0x7fa53c1f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa542453700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c1f4000 0x7fa53c0c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa541c52700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa528004430 0x7fa53c14e120 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa52c001aa0 tx=0x7fa52c0075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa5327fc700 1 -- 172.21.15.12:0/4149962487 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0077d0 con 0x7fa528004430 2023-07-08T12:38:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa5327fc700 1 -- 172.21.15.12:0/4149962487 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c00f8a0 con 0x7fa528004430 2023-07-08T12:38:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa543eb6700 1 -- 172.21.15.12:0/4149962487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53c0cc7e0 con 0x7fa528004430 2023-07-08T12:38:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa5327fc700 1 -- 172.21.15.12:0/4149962487 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00fc50 con 0x7fa528004430 2023-07-08T12:38:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.943+0000 7fa543eb6700 1 -- 172.21.15.12:0/4149962487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53c0c7220 con 0x7fa528004430 2023-07-08T12:38:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.944+0000 7fa543eb6700 1 -- 172.21.15.12:0/4149962487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53c04ea60 con 0x7fa528004430 2023-07-08T12:38:55.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.945+0000 7fa5327fc700 1 -- 172.21.15.12:0/4149962487 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa52c01d460 con 0x7fa528004430 2023-07-08T12:38:55.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.945+0000 7fa5327fc700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa52006ace0 0x7fa52006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:55.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.945+0000 7fa5327fc700 1 -- 172.21.15.12:0/4149962487 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa52c089840 con 0x7fa528004430 2023-07-08T12:38:55.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.945+0000 7fa541451700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa52006ace0 0x7fa52006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:55.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.946+0000 7fa541451700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa52006ace0 0x7fa52006d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa53c1f3990 tx=0x7fa538008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:55.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:55.950+0000 7fa5327fc700 1 -- 172.21.15.12:0/4149962487 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa52c0545b0 con 0x7fa528004430 2023-07-08T12:38:56.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.148+0000 7fa543eb6700 1 -- 172.21.15.12:0/4149962487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa53c0d0920 con 0x7fa528004430 2023-07-08T12:38:56.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.148+0000 7fa5327fc700 1 -- 172.21.15.12:0/4149962487 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa52c057bd0 con 0x7fa528004430 2023-07-08T12:38:56.152 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-07-08T12:38:56.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.150+0000 7fa543eb6700 1 -- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa52006ace0 msgr2=0x7fa52006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:56.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.150+0000 7fa543eb6700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa52006ace0 0x7fa52006d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa53c1f3990 tx=0x7fa538008030 comp rx=0 tx=0).stop 2023-07-08T12:38:56.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.150+0000 7fa543eb6700 1 -- 172.21.15.12:0/4149962487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa528004430 msgr2=0x7fa53c14e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:56.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.150+0000 7fa543eb6700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa528004430 0x7fa53c14e120 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa52c001aa0 tx=0x7fa52c0075f0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.150+0000 7fa543eb6700 1 -- 172.21.15.12:0/4149962487 shutdown_connections 2023-07-08T12:38:56.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.150+0000 7fa543eb6700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa52006ace0 0x7fa52006d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.150+0000 7fa543eb6700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa53c1f4000 0x7fa53c0c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.150+0000 7fa543eb6700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa528004430 0x7fa53c14e120 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.150+0000 7fa543eb6700 1 --2- 172.21.15.12:0/4149962487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa53c0c5d10 0x7fa53c1f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.150+0000 7fa543eb6700 1 -- 172.21.15.12:0/4149962487 >> 172.21.15.12:0/4149962487 conn(0x7fa53c099c60 msgr2=0x7fa53c159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:56.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.150+0000 7fa543eb6700 1 -- 172.21.15.12:0/4149962487 shutdown_connections 2023-07-08T12:38:56.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.150+0000 7fa543eb6700 1 -- 172.21.15.12:0/4149962487 wait complete. 2023-07-08T12:38:56.165 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-08T12:38:56.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.291+0000 7f70b2973700 1 Processor -- start 2023-07-08T12:38:56.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.291+0000 7f70b2973700 1 -- start start 2023-07-08T12:38:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.292+0000 7f70b2973700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f70ac0aeec0 0x7f70ac06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.292+0000 7f70b2973700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ac06ebe0 0x7f70ac0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.292+0000 7f70b2973700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70ac0cd9b0 0x7f70ac0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.292+0000 7f70b2973700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ac15d770 con 0x7f70ac0cd9b0 2023-07-08T12:38:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.292+0000 7f70b2973700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ac0cddd0 con 0x7f70ac0aeec0 2023-07-08T12:38:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.292+0000 7f70b2973700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac15cb60 con 0x7f70ac06ebe0 2023-07-08T12:38:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.292+0000 7f70b0f10700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70ac0cd9b0 0x7f70ac0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.292+0000 7f70b0f10700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70ac0cd9b0 0x7f70ac0cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50458/0 (socket says 172.21.15.12:50458) 2023-07-08T12:38:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.292+0000 7f70b0f10700 1 -- 172.21.15.12:0/2476221216 learned_addr learned my addr 172.21.15.12:0/2476221216 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.292+0000 7f70abfff700 1 --2- 172.21.15.12:0/2476221216 >> v2:172.21.15.160:3300/0 conn(0x7f70ac0aeec0 0x7f70ac06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:56.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.292+0000 7f70ab7fe700 1 --2- 172.21.15.12:0/2476221216 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ac06ebe0 0x7f70ac0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:56.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b0f10700 1 -- 172.21.15.12:0/2476221216 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ac06ebe0 msgr2=0x7f70ac0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:56.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b0f10700 1 --2- 172.21.15.12:0/2476221216 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ac06ebe0 0x7f70ac0cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b0f10700 1 -- 172.21.15.12:0/2476221216 >> v2:172.21.15.160:3300/0 conn(0x7f70ac0aeec0 msgr2=0x7f70ac06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:56.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b0f10700 1 --2- 172.21.15.12:0/2476221216 >> v2:172.21.15.160:3300/0 conn(0x7f70ac0aeec0 0x7f70ac06cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b0f10700 1 -- 172.21.15.12:0/2476221216 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac15ce40 con 0x7f70ac0cd9b0 2023-07-08T12:38:56.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70ab7fe700 1 --2- 172.21.15.12:0/2476221216 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ac06ebe0 0x7f70ac0cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:56.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70abfff700 1 --2- 172.21.15.12:0/2476221216 >> v2:172.21.15.160:3300/0 conn(0x7f70ac0aeec0 0x7f70ac06cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:56.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b0f10700 1 --2- 172.21.15.12:0/2476221216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ac0cd9b0 0x7f70ac0cbe10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0b23f0 tx=0x7f70a400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660b07c2242a34fe server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70aaffd700 1 -- 172.21.15.12:0/2476221216 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4010070 con 0x7f70ac0cd9b0 2023-07-08T12:38:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70aaffd700 1 -- 172.21.15.12:0/2476221216 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a400dd90 con 0x7f70ac0cd9b0 2023-07-08T12:38:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70aaffd700 1 -- 172.21.15.12:0/2476221216 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400b310 con 0x7f70ac0cd9b0 2023-07-08T12:38:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b2973700 1 -- 172.21.15.12:0/2476221216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ac0cd9b0 msgr2=0x7f70ac0cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:56.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b2973700 1 --2- 172.21.15.12:0/2476221216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ac0cd9b0 0x7f70ac0cbe10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0b23f0 tx=0x7f70a400d430 comp rx=0 tx=0).stop 2023-07-08T12:38:56.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b2973700 1 -- 172.21.15.12:0/2476221216 shutdown_connections 2023-07-08T12:38:56.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b2973700 1 --2- 172.21.15.12:0/2476221216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ac0cd9b0 0x7f70ac0cbe10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b2973700 1 --2- 172.21.15.12:0/2476221216 >> v2:172.21.15.160:3300/0 conn(0x7f70ac0aeec0 0x7f70ac06cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b2973700 1 --2- 172.21.15.12:0/2476221216 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ac06ebe0 0x7f70ac0cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b2973700 1 -- 172.21.15.12:0/2476221216 >> 172.21.15.12:0/2476221216 conn(0x7f70ac098b50 msgr2=0x7f70ac06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:56.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b2973700 1 -- 172.21.15.12:0/2476221216 shutdown_connections 2023-07-08T12:38:56.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.293+0000 7f70b2973700 1 -- 172.21.15.12:0/2476221216 wait complete. 2023-07-08T12:38:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.295+0000 7f70b2973700 1 Processor -- start 2023-07-08T12:38:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.295+0000 7f70b2973700 1 -- start start 2023-07-08T12:38:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.296+0000 7f70b2973700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ac06ebe0 0x7f70ac1557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.296+0000 7f70b2973700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ac0aeec0 0x7f70ac157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:56.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.296+0000 7f70b2973700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70ac0cd9b0 0x7f70ac157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:56.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.296+0000 7f70b2973700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac0c9db0 con 0x7f70ac0aeec0 2023-07-08T12:38:56.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.296+0000 7f70b2973700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac0ca930 con 0x7f70ac0cd9b0 2023-07-08T12:38:56.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.296+0000 7f70b2973700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac0ca000 con 0x7f70ac06ebe0 2023-07-08T12:38:56.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.296+0000 7f70ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ac0aeec0 0x7f70ac157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:56.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.296+0000 7f70ab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ac0aeec0 0x7f70ac157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50462/0 (socket says 172.21.15.12:50462) 2023-07-08T12:38:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.296+0000 7f70ab7fe700 1 -- 172.21.15.12:0/1020142789 learned_addr learned my addr 172.21.15.12:0/1020142789 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.296+0000 7f70abfff700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ac06ebe0 0x7f70ac1557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.296+0000 7f70b0f10700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70ac0cd9b0 0x7f70ac157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.297+0000 7f70ab7fe700 1 -- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ac06ebe0 msgr2=0x7f70ac1557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:56.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.297+0000 7f70ab7fe700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ac06ebe0 0x7f70ac1557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.297+0000 7f70ab7fe700 1 -- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70ac0cd9b0 msgr2=0x7f70ac157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:56.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.297+0000 7f70ab7fe700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70ac0cd9b0 0x7f70ac157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.297+0000 7f70ab7fe700 1 -- 172.21.15.12:0/1020142789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a400bd40 con 0x7f70ac0aeec0 2023-07-08T12:38:56.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.297+0000 7f70ab7fe700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ac0aeec0 0x7f70ac157a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f709400d800 tx=0x7f709400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:56.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.297+0000 7f70a8ff9700 1 -- 172.21.15.12:0/1020142789 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094004030 con 0x7f70ac0aeec0 2023-07-08T12:38:56.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.297+0000 7f70a8ff9700 1 -- 172.21.15.12:0/1020142789 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7094004190 con 0x7f70ac0aeec0 2023-07-08T12:38:56.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.297+0000 7f70b2973700 1 -- 172.21.15.12:0/1020142789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ac0ca2e0 con 0x7f70ac0aeec0 2023-07-08T12:38:56.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.297+0000 7f70a8ff9700 1 -- 172.21.15.12:0/1020142789 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094004520 con 0x7f70ac0aeec0 2023-07-08T12:38:56.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.297+0000 7f70b2973700 1 -- 172.21.15.12:0/1020142789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ac155d00 con 0x7f70ac0aeec0 2023-07-08T12:38:56.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.298+0000 7f70b2973700 1 -- 172.21.15.12:0/1020142789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ac0caaa0 con 0x7f70ac0aeec0 2023-07-08T12:38:56.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.298+0000 7f70a8ff9700 1 -- 172.21.15.12:0/1020142789 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70940036a0 con 0x7f70ac0aeec0 2023-07-08T12:38:56.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.299+0000 7f70a8ff9700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f708c06ace0 0x7f708c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:56.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.299+0000 7f70a8ff9700 1 -- 172.21.15.12:0/1020142789 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7094088f10 con 0x7f70ac0aeec0 2023-07-08T12:38:56.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.299+0000 7f70abfff700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f708c06ace0 0x7f708c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:56.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.300+0000 7f70abfff700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f708c06ace0 0x7f708c06d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0ace30 tx=0x7f709c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:56.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.303+0000 7f70a8ff9700 1 -- 172.21.15.12:0/1020142789 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7094057860 con 0x7f70ac0aeec0 2023-07-08T12:38:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.506+0000 7f70b2973700 1 -- 172.21.15.12:0/1020142789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f70ac04ea60 con 0x7f70ac0aeec0 2023-07-08T12:38:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.507+0000 7f70a8ff9700 1 -- 172.21.15.12:0/1020142789 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f70940190a0 con 0x7f70ac0aeec0 2023-07-08T12:38:56.509 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:38:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.509+0000 7f70b2973700 1 -- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f708c06ace0 msgr2=0x7f708c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.509+0000 7f70b2973700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f708c06ace0 0x7f708c06d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0ace30 tx=0x7f709c008040 comp rx=0 tx=0).stop 2023-07-08T12:38:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.509+0000 7f70b2973700 1 -- 172.21.15.12:0/1020142789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ac0aeec0 msgr2=0x7f70ac157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.509+0000 7f70b2973700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ac0aeec0 0x7f70ac157a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f709400d800 tx=0x7f709400db10 comp rx=0 tx=0).stop 2023-07-08T12:38:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.509+0000 7f70b2973700 1 -- 172.21.15.12:0/1020142789 shutdown_connections 2023-07-08T12:38:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.509+0000 7f70b2973700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f708c06ace0 0x7f708c06d190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.509+0000 7f70b2973700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70ac0cd9b0 0x7f70ac157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.509+0000 7f70b2973700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ac06ebe0 0x7f70ac1557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.509+0000 7f70b2973700 1 --2- 172.21.15.12:0/1020142789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ac0aeec0 0x7f70ac157a20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.509+0000 7f70b2973700 1 -- 172.21.15.12:0/1020142789 >> 172.21.15.12:0/1020142789 conn(0x7f70ac098b50 msgr2=0x7f70ac15b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.509+0000 7f70b2973700 1 -- 172.21.15.12:0/1020142789 shutdown_connections 2023-07-08T12:38:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.509+0000 7f70b2973700 1 -- 172.21.15.12:0/1020142789 wait complete. 2023-07-08T12:38:56.523 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:56.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.648+0000 7f56af85e700 1 Processor -- start 2023-07-08T12:38:56.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.648+0000 7f56af85e700 1 -- start start 2023-07-08T12:38:56.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.649+0000 7f56af85e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a80b8320 0x7f56a80cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:56.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.649+0000 7f56af85e700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f56a80d1cd0 0x7f56a81608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:56.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.649+0000 7f56af85e700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a80cf3b0 con 0x7f56a80b45b0 2023-07-08T12:38:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.649+0000 7f56af85e700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a8160e20 con 0x7f56a80d1cd0 2023-07-08T12:38:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.649+0000 7f56af85e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a80b86f0 con 0x7f56a80b8320 2023-07-08T12:38:56.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.649+0000 7f56acdf9700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f56a80d1cd0 0x7f56a81608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:56.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.649+0000 7f56acdf9700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f56a80d1cd0 0x7f56a81608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50468/0 (socket says 172.21.15.12:50468) 2023-07-08T12:38:56.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.649+0000 7f56addfb700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f56a80b45b0 0x7f56a80d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37868/0 (socket says 172.21.15.12:37868) 2023-07-08T12:38:56.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.649+0000 7f56acdf9700 1 -- 172.21.15.12:0/1688426320 learned_addr learned my addr 172.21.15.12:0/1688426320 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:56.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.650+0000 7f56ad5fa700 1 --2- 172.21.15.12:0/1688426320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a80b8320 0x7f56a80cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:56.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.650+0000 7f56acdf9700 1 -- 172.21.15.12:0/1688426320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a80b8320 msgr2=0x7f56a80cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:56.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.650+0000 7f56acdf9700 1 --2- 172.21.15.12:0/1688426320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a80b8320 0x7f56a80cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.650+0000 7f56acdf9700 1 -- 172.21.15.12:0/1688426320 >> v1:172.21.15.12:6789/0 conn(0x7f56a80b45b0 legacy=0x7f56a80d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:56.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.650+0000 7f56acdf9700 1 -- 172.21.15.12:0/1688426320 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a80cf0d0 con 0x7f56a80d1cd0 2023-07-08T12:38:56.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.650+0000 7f569ffff700 1 -- 172.21.15.12:0/1688426320 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908482917 0 0) 0x7f56a80cf3b0 con 0x7f56a80b45b0 2023-07-08T12:38:56.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.650+0000 7f56acdf9700 1 --2- 172.21.15.12:0/1688426320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56a80d1cd0 0x7f56a81608e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f56a40098f0 tx=0x7f56a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed10452c5a1d89ab server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:56.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.650+0000 7f569ffff700 1 -- 172.21.15.12:0/1688426320 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a400b8a0 con 0x7f56a80d1cd0 2023-07-08T12:38:56.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.650+0000 7f569ffff700 1 -- 172.21.15.12:0/1688426320 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a400ba00 con 0x7f56a80d1cd0 2023-07-08T12:38:56.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.650+0000 7f569ffff700 1 -- 172.21.15.12:0/1688426320 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a400bd90 con 0x7f56a80d1cd0 2023-07-08T12:38:56.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.651+0000 7f56af85e700 1 -- 172.21.15.12:0/1688426320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56a80d1cd0 msgr2=0x7f56a81608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:56.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.651+0000 7f56af85e700 1 --2- 172.21.15.12:0/1688426320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56a80d1cd0 0x7f56a81608e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f56a40098f0 tx=0x7f56a4009c00 comp rx=0 tx=0).stop 2023-07-08T12:38:56.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.651+0000 7f56af85e700 1 -- 172.21.15.12:0/1688426320 shutdown_connections 2023-07-08T12:38:56.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.651+0000 7f56af85e700 1 --2- 172.21.15.12:0/1688426320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a80b8320 0x7f56a80cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.651+0000 7f56af85e700 1 --2- 172.21.15.12:0/1688426320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56a80d1cd0 0x7f56a81608e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.651+0000 7f56af85e700 1 -- 172.21.15.12:0/1688426320 >> 172.21.15.12:0/1688426320 conn(0x7f56a80722b0 msgr2=0x7f56a80cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.651+0000 7f56af85e700 1 -- 172.21.15.12:0/1688426320 shutdown_connections 2023-07-08T12:38:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.651+0000 7f56af85e700 1 -- 172.21.15.12:0/1688426320 wait complete. 2023-07-08T12:38:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.652+0000 7f56af85e700 1 Processor -- start 2023-07-08T12:38:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.652+0000 7f56af85e700 1 -- start start 2023-07-08T12:38:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56af85e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56a80b8320 0x7f56a815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56af85e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a80d1cd0 0x7f56a8146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56af85e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56a80b7790 0x7f56a815d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:56.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56af85e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81473c0 con 0x7f56a80b7790 2023-07-08T12:38:56.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56af85e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a80b66f0 con 0x7f56a80b8320 2023-07-08T12:38:56.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56af85e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a80b69d0 con 0x7f56a80d1cd0 2023-07-08T12:38:56.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56addfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56a80b7790 0x7f56a815d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:56.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56addfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56a80b7790 0x7f56a815d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50478/0 (socket says 172.21.15.12:50478) 2023-07-08T12:38:56.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56addfb700 1 -- 172.21.15.12:0/2083319097 learned_addr learned my addr 172.21.15.12:0/2083319097 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:56.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56acdf9700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a80d1cd0 0x7f56a8146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56ad5fa700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56a80b8320 0x7f56a815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56addfb700 1 -- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a80d1cd0 msgr2=0x7f56a8146e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56addfb700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a80d1cd0 0x7f56a8146e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56addfb700 1 -- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56a80b8320 msgr2=0x7f56a815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56addfb700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56a80b8320 0x7f56a815c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56addfb700 1 -- 172.21.15.12:0/2083319097 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a4009550 con 0x7f56a80b7790 2023-07-08T12:38:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f56addfb700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56a80b7790 0x7f56a815d600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f56a0002230 tx=0x7f56a000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.653+0000 7f569dffb700 1 -- 172.21.15.12:0/2083319097 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a000ac20 con 0x7f56a80b7790 2023-07-08T12:38:56.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.654+0000 7f569dffb700 1 -- 172.21.15.12:0/2083319097 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a000ada0 con 0x7f56a80b7790 2023-07-08T12:38:56.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.654+0000 7f56af85e700 1 -- 172.21.15.12:0/2083319097 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a80b6cb0 con 0x7f56a80b7790 2023-07-08T12:38:56.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.654+0000 7f569dffb700 1 -- 172.21.15.12:0/2083319097 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0007310 con 0x7f56a80b7790 2023-07-08T12:38:56.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.654+0000 7f56af85e700 1 -- 172.21.15.12:0/2083319097 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a80b7270 con 0x7f56a80b7790 2023-07-08T12:38:56.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.655+0000 7f56af85e700 1 -- 172.21.15.12:0/2083319097 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a80b7460 con 0x7f56a80b7790 2023-07-08T12:38:56.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.655+0000 7f569dffb700 1 -- 172.21.15.12:0/2083319097 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f56a0007470 con 0x7f56a80b7790 2023-07-08T12:38:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.655+0000 7f569dffb700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f568c06ace0 0x7f568c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.656+0000 7f569dffb700 1 -- 172.21.15.12:0/2083319097 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f56a0089640 con 0x7f56a80b7790 2023-07-08T12:38:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.656+0000 7f56ad5fa700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f568c06ace0 0x7f568c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.656+0000 7f56ad5fa700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f568c06ace0 0x7f568c06d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5698009930 tx=0x7f5698008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:56.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.660+0000 7f569dffb700 1 -- 172.21.15.12:0/2083319097 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f56a0054450 con 0x7f56a80b7790 2023-07-08T12:38:56.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.858+0000 7f56af85e700 1 -- 172.21.15.12:0/2083319097 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56a80d2290 con 0x7f56a80b7790 2023-07-08T12:38:56.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.859+0000 7f569dffb700 1 -- 172.21.15.12:0/2083319097 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20203 (secure 0 0 0) 0x7f56a0057a70 con 0x7f56a80b7790 2023-07-08T12:38:56.862 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:56.863 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:49.386462+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:56.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.862+0000 7f56af85e700 1 -- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f568c06ace0 msgr2=0x7f568c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:56.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.862+0000 7f56af85e700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f568c06ace0 0x7f568c06d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5698009930 tx=0x7f5698008040 comp rx=0 tx=0).stop 2023-07-08T12:38:56.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.862+0000 7f56af85e700 1 -- 172.21.15.12:0/2083319097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56a80b7790 msgr2=0x7f56a815d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:56.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.862+0000 7f56af85e700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56a80b7790 0x7f56a815d600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f56a0002230 tx=0x7f56a000a060 comp rx=0 tx=0).stop 2023-07-08T12:38:56.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.862+0000 7f56af85e700 1 -- 172.21.15.12:0/2083319097 shutdown_connections 2023-07-08T12:38:56.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.862+0000 7f56af85e700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f568c06ace0 0x7f568c06d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.862+0000 7f56af85e700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56a80b7790 0x7f56a815d600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.862+0000 7f56af85e700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56a80b8320 0x7f56a815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.862+0000 7f56af85e700 1 --2- 172.21.15.12:0/2083319097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a80d1cd0 0x7f56a8146e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:56.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.862+0000 7f56af85e700 1 -- 172.21.15.12:0/2083319097 >> 172.21.15.12:0/2083319097 conn(0x7f56a80b45b0 msgr2=0x7f56a806b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:56.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.862+0000 7f56af85e700 1 -- 172.21.15.12:0/2083319097 shutdown_connections 2023-07-08T12:38:56.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:56.862+0000 7f56af85e700 1 -- 172.21.15.12:0/2083319097 wait complete. 2023-07-08T12:38:56.877 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-08T12:38:05.660194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-08T12:38:06.920177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-08T12:38:10.592749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-08T12:38:56.878 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:38:57.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.004+0000 7fabd483c700 1 Processor -- start 2023-07-08T12:38:57.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.004+0000 7fabd483c700 1 -- start start 2023-07-08T12:38:57.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.005+0000 7fabd483c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd0156a10 0x7fabd006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.005+0000 7fabd483c700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fabd00cc240 0x7fabd00cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.005+0000 7fabd483c700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd0160410 con 0x7fabd015d390 2023-07-08T12:38:57.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.005+0000 7fabd483c700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd00ccc00 con 0x7fabd00cc240 2023-07-08T12:38:57.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.005+0000 7fabd483c700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd0156de0 con 0x7fabd0156a10 2023-07-08T12:38:57.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.005+0000 7fabcdd9b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fabd015d390 0x7fabd01608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37880/0 (socket says 172.21.15.12:37880) 2023-07-08T12:38:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.005+0000 7fabcdd9b700 1 -- 172.21.15.12:0/3023324722 learned_addr learned my addr 172.21.15.12:0/3023324722 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.005+0000 7fabce59c700 1 --2- 172.21.15.12:0/3023324722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd0156a10 0x7fabd006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.005+0000 7fabced9d700 1 --2- 172.21.15.12:0/3023324722 >> v2:172.21.15.160:3300/0 conn(0x7fabd00cc240 0x7fabd00cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.005+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764624173 0 0) 0x7fabd0160410 con 0x7fabd015d390 2023-07-08T12:38:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb80035b0 con 0x7fabd015d390 2023-07-08T12:38:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3011603448 0 0) 0x7fabb80035b0 con 0x7fabd015d390 2023-07-08T12:38:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd0156a10 msgr2=0x7fabd006e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:57.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 --2- 172.21.15.12:0/3023324722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd0156a10 0x7fabd006e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 >> v2:172.21.15.160:3300/0 conn(0x7fabd00cc240 msgr2=0x7fabd00cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:57.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 --2- 172.21.15.12:0/3023324722 >> v2:172.21.15.160:3300/0 conn(0x7fabd00cc240 0x7fabd00cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd006ec70 con 0x7fabd015d390 2023-07-08T12:38:57.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7fabc0001670 con 0x7fabd015d390 2023-07-08T12:38:57.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 >> v1:172.21.15.12:6789/0 conn(0x7fabd015d390 legacy=0x7fabd01608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 --2- 172.21.15.12:0/3023324722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd0156a10 0x7fabb8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 --2- 172.21.15.12:0/3023324722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabb8003f80 0x7fabb8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 --2- 172.21.15.12:0/3023324722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabb8004d90 0x7fabb8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd00ccc00 con 0x7fabb8004d90 2023-07-08T12:38:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd0156de0 con 0x7fabb8003f80 2023-07-08T12:38:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb800ba10 con 0x7fabd0156a10 2023-07-08T12:38:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabced9d700 1 --2- 172.21.15.12:0/3023324722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabb8004d90 0x7fabb8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabce59c700 1 --2- 172.21.15.12:0/3023324722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd0156a10 0x7fabb8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabcdd9b700 1 --2- 172.21.15.12:0/3023324722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabb8003f80 0x7fabb8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabced9d700 1 -- 172.21.15.12:0/3023324722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd0156a10 msgr2=0x7fabb8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabced9d700 1 --2- 172.21.15.12:0/3023324722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd0156a10 0x7fabb8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabced9d700 1 -- 172.21.15.12:0/3023324722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabb8003f80 msgr2=0x7fabb8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabced9d700 1 --2- 172.21.15.12:0/3023324722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabb8003f80 0x7fabb8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.006+0000 7fabced9d700 1 -- 172.21.15.12:0/3023324722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb80035b0 con 0x7fabb8004d90 2023-07-08T12:38:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.007+0000 7fabced9d700 1 --2- 172.21.15.12:0/3023324722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabb8004d90 0x7fabb8005200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009fc0 tx=0x7fabb8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ddcacd50e41ffc server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.007+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc800ec30 con 0x7fabb8004d90 2023-07-08T12:38:57.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.007+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc800f040 con 0x7fabb8004d90 2023-07-08T12:38:57.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.007+0000 7fabcd59a700 1 -- 172.21.15.12:0/3023324722 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8014480 con 0x7fabb8004d90 2023-07-08T12:38:57.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.007+0000 7fabd483c700 1 -- 172.21.15.12:0/3023324722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabb8004d90 msgr2=0x7fabb8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.007+0000 7fabd483c700 1 --2- 172.21.15.12:0/3023324722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabb8004d90 0x7fabb8005200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009fc0 tx=0x7fabb8005880 comp rx=0 tx=0).stop 2023-07-08T12:38:57.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.007+0000 7fabced9d700 1 -- 172.21.15.12:0/3023324722 reap_dead start 2023-07-08T12:38:57.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.007+0000 7fabd483c700 1 -- 172.21.15.12:0/3023324722 shutdown_connections 2023-07-08T12:38:57.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.007+0000 7fabd483c700 1 -- 172.21.15.12:0/3023324722 >> 172.21.15.12:0/3023324722 conn(0x7fabd0098b50 msgr2=0x7fabd006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:57.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.007+0000 7fabd483c700 1 -- 172.21.15.12:0/3023324722 shutdown_connections 2023-07-08T12:38:57.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.007+0000 7fabd483c700 1 -- 172.21.15.12:0/3023324722 wait complete. 2023-07-08T12:38:57.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.009+0000 7fabd483c700 1 Processor -- start 2023-07-08T12:38:57.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.009+0000 7fabd483c700 1 -- start start 2023-07-08T12:38:57.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.009+0000 7fabd483c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabb8004d90 0x7fabd01573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.009+0000 7fabd483c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd015e980 0x7fabd015edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.009+0000 7fabd483c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabd01448d0 0x7fabd0144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabd483c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd015f330 con 0x7fabb8004d90 2023-07-08T12:38:57.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabd483c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd01426d0 con 0x7fabd01448d0 2023-07-08T12:38:57.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabd483c700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd01e5e00 con 0x7fabd015e980 2023-07-08T12:38:57.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabce59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabb8004d90 0x7fabd01573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabce59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabb8004d90 0x7fabd01573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50494/0 (socket says 172.21.15.12:50494) 2023-07-08T12:38:57.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabce59c700 1 -- 172.21.15.12:0/2036557419 learned_addr learned my addr 172.21.15.12:0/2036557419 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:57.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabcdd9b700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd015e980 0x7fabd015edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabced9d700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabd01448d0 0x7fabd0144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabce59c700 1 -- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd015e980 msgr2=0x7fabd015edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabce59c700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd015e980 0x7fabd015edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabce59c700 1 -- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabd01448d0 msgr2=0x7fabd0144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabce59c700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabd01448d0 0x7fabd0144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabce59c700 1 -- 172.21.15.12:0/2036557419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd01e60e0 con 0x7fabb8004d90 2023-07-08T12:38:57.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabce59c700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabb8004d90 0x7fabd01573f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fabc4011a20 tx=0x7fabc4011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:57.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabbeffd700 1 -- 172.21.15.12:0/2036557419 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc4005c30 con 0x7fabb8004d90 2023-07-08T12:38:57.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabbeffd700 1 -- 172.21.15.12:0/2036557419 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc4005d90 con 0x7fabb8004d90 2023-07-08T12:38:57.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabd483c700 1 -- 172.21.15.12:0/2036557419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd01e63c0 con 0x7fabb8004d90 2023-07-08T12:38:57.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabbeffd700 1 -- 172.21.15.12:0/2036557419 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc400fa10 con 0x7fabb8004d90 2023-07-08T12:38:57.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.010+0000 7fabd483c700 1 -- 172.21.15.12:0/2036557419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd0145280 con 0x7fabb8004d90 2023-07-08T12:38:57.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.011+0000 7fabd483c700 1 -- 172.21.15.12:0/2036557419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd0142c90 con 0x7fabb8004d90 2023-07-08T12:38:57.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.012+0000 7fabbeffd700 1 -- 172.21.15.12:0/2036557419 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fabc400fb70 con 0x7fabb8004d90 2023-07-08T12:38:57.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.012+0000 7fabbeffd700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabb006ace0 0x7fabb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.012+0000 7fabbeffd700 1 -- 172.21.15.12:0/2036557419 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fabc4089f90 con 0x7fabb8004d90 2023-07-08T12:38:57.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.012+0000 7fabcdd9b700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabb006ace0 0x7fabb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.013+0000 7fabcdd9b700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabb006ace0 0x7fabb006d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fabd01f56e0 tx=0x7fabc0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:57.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.017+0000 7fabbeffd700 1 -- 172.21.15.12:0/2036557419 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fabc40bc310 con 0x7fabb8004d90 2023-07-08T12:38:57.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.215+0000 7fabd483c700 1 -- 172.21.15.12:0/2036557419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabd0066af0 con 0x7fabb8004d90 2023-07-08T12:38:57.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.215+0000 7fabbeffd700 1 -- 172.21.15.12:0/2036557419 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabc4054e80 con 0x7fabb8004d90 2023-07-08T12:38:57.218 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:38:57.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.217+0000 7fabd483c700 1 -- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabb006ace0 msgr2=0x7fabb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.217+0000 7fabd483c700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabb006ace0 0x7fabb006d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fabd01f56e0 tx=0x7fabc0009040 comp rx=0 tx=0).stop 2023-07-08T12:38:57.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.217+0000 7fabd483c700 1 -- 172.21.15.12:0/2036557419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabb8004d90 msgr2=0x7fabd01573f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.217+0000 7fabd483c700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabb8004d90 0x7fabd01573f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fabc4011a20 tx=0x7fabc4011de0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.217+0000 7fabd483c700 1 -- 172.21.15.12:0/2036557419 shutdown_connections 2023-07-08T12:38:57.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.217+0000 7fabd483c700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabb006ace0 0x7fabb006d190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.217+0000 7fabd483c700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabd01448d0 0x7fabd0144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.217+0000 7fabd483c700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabb8004d90 0x7fabd01573f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.217+0000 7fabd483c700 1 --2- 172.21.15.12:0/2036557419 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabd015e980 0x7fabd015edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.217+0000 7fabd483c700 1 -- 172.21.15.12:0/2036557419 >> 172.21.15.12:0/2036557419 conn(0x7fabd0098b50 msgr2=0x7fabd00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:57.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.217+0000 7fabd483c700 1 -- 172.21.15.12:0/2036557419 shutdown_connections 2023-07-08T12:38:57.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.218+0000 7fabd483c700 1 -- 172.21.15.12:0/2036557419 wait complete. 2023-07-08T12:38:57.232 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:38:57.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.359+0000 7fd5618cf700 1 Processor -- start 2023-07-08T12:38:57.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.359+0000 7fd5618cf700 1 -- start start 2023-07-08T12:38:57.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd5618cf700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd55c0b8430 0x7fd55c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd5618cf700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd55c0d3730 0x7fd55c0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd5618cf700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55c0d0b20 con 0x7fd55c0cfa90 2023-07-08T12:38:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd5618cf700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55c0b8800 con 0x7fd55c0b8430 2023-07-08T12:38:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd5618cf700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0cfeb0 con 0x7fd55c0d3730 2023-07-08T12:38:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd55b7fe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd55c0cfa90 0x7fd55c15feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37902/0 (socket says 172.21.15.12:37902) 2023-07-08T12:38:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd55b7fe700 1 -- 172.21.15.12:0/2333841060 learned_addr learned my addr 172.21.15.12:0/2333841060 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd55affd700 1 --2- 172.21.15.12:0/2333841060 >> v2:172.21.15.160:3300/0 conn(0x7fd55c0b8430 0x7fd55c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd55a7fc700 1 --2- 172.21.15.12:0/2333841060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd55c0d3730 0x7fd55c0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2851525731 0 0) 0x7fd55c0d0b20 con 0x7fd55c0cfa90 2023-07-08T12:38:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5440035b0 con 0x7fd55c0cfa90 2023-07-08T12:38:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1884545443 0 0) 0x7fd5440035b0 con 0x7fd55c0cfa90 2023-07-08T12:38:57.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd55c0d3730 msgr2=0x7fd55c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:57.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd559ffb700 1 --2- 172.21.15.12:0/2333841060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd55c0d3730 0x7fd55c0cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 >> v2:172.21.15.160:3300/0 conn(0x7fd55c0b8430 msgr2=0x7fd55c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:57.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd559ffb700 1 --2- 172.21.15.12:0/2333841060 >> v2:172.21.15.160:3300/0 conn(0x7fd55c0b8430 0x7fd55c0cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c0b4ee0 con 0x7fd55c0cfa90 2023-07-08T12:38:57.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.360+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7fd5540019f0 con 0x7fd55c0cfa90 2023-07-08T12:38:57.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 >> v1:172.21.15.12:6789/0 conn(0x7fd55c0cfa90 legacy=0x7fd55c15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd559ffb700 1 --2- 172.21.15.12:0/2333841060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd544003a00 0x7fd544003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd559ffb700 1 --2- 172.21.15.12:0/2333841060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd55c0d3730 0x7fd544004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd559ffb700 1 --2- 172.21.15.12:0/2333841060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd544004d90 0x7fd544005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0b8800 con 0x7fd544004d90 2023-07-08T12:38:57.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0cfeb0 con 0x7fd544003a00 2023-07-08T12:38:57.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54400ba10 con 0x7fd55c0d3730 2023-07-08T12:38:57.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd55b7fe700 1 --2- 172.21.15.12:0/2333841060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd544004d90 0x7fd544005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd55a7fc700 1 --2- 172.21.15.12:0/2333841060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd55c0d3730 0x7fd544004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd55affd700 1 --2- 172.21.15.12:0/2333841060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd544003a00 0x7fd544003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd55b7fe700 1 -- 172.21.15.12:0/2333841060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd55c0d3730 msgr2=0x7fd544004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:57.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd55b7fe700 1 --2- 172.21.15.12:0/2333841060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd55c0d3730 0x7fd544004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd55b7fe700 1 -- 172.21.15.12:0/2333841060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd544003a00 msgr2=0x7fd544003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:57.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd55b7fe700 1 --2- 172.21.15.12:0/2333841060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd544003a00 0x7fd544003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd55b7fe700 1 -- 172.21.15.12:0/2333841060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440035b0 con 0x7fd544004d90 2023-07-08T12:38:57.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd55b7fe700 1 --2- 172.21.15.12:0/2333841060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd544004d90 0x7fd544005200 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd55400a850 tx=0x7fd544005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3645cd41691904 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:57.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554012600 con 0x7fd544004d90 2023-07-08T12:38:57.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd554012780 con 0x7fd544004d90 2023-07-08T12:38:57.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.361+0000 7fd559ffb700 1 -- 172.21.15.12:0/2333841060 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554012b30 con 0x7fd544004d90 2023-07-08T12:38:57.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.362+0000 7fd5618cf700 1 -- 172.21.15.12:0/2333841060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd544004d90 msgr2=0x7fd544005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.362+0000 7fd5618cf700 1 --2- 172.21.15.12:0/2333841060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd544004d90 0x7fd544005200 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd55400a850 tx=0x7fd544005880 comp rx=0 tx=0).stop 2023-07-08T12:38:57.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.362+0000 7fd55b7fe700 1 -- 172.21.15.12:0/2333841060 reap_dead start 2023-07-08T12:38:57.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.362+0000 7fd5618cf700 1 -- 172.21.15.12:0/2333841060 shutdown_connections 2023-07-08T12:38:57.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.362+0000 7fd5618cf700 1 -- 172.21.15.12:0/2333841060 >> 172.21.15.12:0/2333841060 conn(0x7fd55c06b3f0 msgr2=0x7fd55c06b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:57.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.362+0000 7fd5618cf700 1 -- 172.21.15.12:0/2333841060 shutdown_connections 2023-07-08T12:38:57.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.362+0000 7fd5618cf700 1 -- 172.21.15.12:0/2333841060 wait complete. 2023-07-08T12:38:57.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.363+0000 7fd5618cf700 1 Processor -- start 2023-07-08T12:38:57.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.363+0000 7fd5618cf700 1 -- start start 2023-07-08T12:38:57.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.364+0000 7fd5618cf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd544004d90 0x7fd55c155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.364+0000 7fd5618cf700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd55c0ca620 0x7fd55c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.364+0000 7fd5618cf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd55c0c7900 0x7fd55c0c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.364+0000 7fd5618cf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c155640 con 0x7fd55c0c7900 2023-07-08T12:38:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.364+0000 7fd5618cf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0c82e0 con 0x7fd544004d90 2023-07-08T12:38:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.364+0000 7fd5618cf700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0cb070 con 0x7fd55c0ca620 2023-07-08T12:38:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.364+0000 7fd55b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd55c0c7900 0x7fd55c0c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.364+0000 7fd55b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd55c0c7900 0x7fd55c0c7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50518/0 (socket says 172.21.15.12:50518) 2023-07-08T12:38:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.364+0000 7fd55a7fc700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd55c0ca620 0x7fd55c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.364+0000 7fd55b7fe700 1 -- 172.21.15.12:0/1774856386 learned_addr learned my addr 172.21.15.12:0/1774856386 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.364+0000 7fd55affd700 1 --2- 172.21.15.12:0/1774856386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd544004d90 0x7fd55c155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.365+0000 7fd55b7fe700 1 -- 172.21.15.12:0/1774856386 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd55c0ca620 msgr2=0x7fd55c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.365+0000 7fd55b7fe700 1 --2- 172.21.15.12:0/1774856386 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd55c0ca620 0x7fd55c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.365+0000 7fd55b7fe700 1 -- 172.21.15.12:0/1774856386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd544004d90 msgr2=0x7fd55c155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.365+0000 7fd55b7fe700 1 --2- 172.21.15.12:0/1774856386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd544004d90 0x7fd55c155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.365+0000 7fd55b7fe700 1 -- 172.21.15.12:0/1774856386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c0cb350 con 0x7fd55c0c7900 2023-07-08T12:38:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.365+0000 7fd55b7fe700 1 --2- 172.21.15.12:0/1774856386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd55c0c7900 0x7fd55c0c7da0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd554012000 tx=0x7fd554001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.365+0000 7fd543fff700 1 -- 172.21.15.12:0/1774856386 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55401e630 con 0x7fd55c0c7900 2023-07-08T12:38:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.365+0000 7fd543fff700 1 -- 172.21.15.12:0/1774856386 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55401e790 con 0x7fd55c0c7900 2023-07-08T12:38:57.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.365+0000 7fd5618cf700 1 -- 172.21.15.12:0/1774856386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55c0cb630 con 0x7fd55c0c7900 2023-07-08T12:38:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.365+0000 7fd543fff700 1 -- 172.21.15.12:0/1774856386 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55401eb40 con 0x7fd55c0c7900 2023-07-08T12:38:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.365+0000 7fd5618cf700 1 -- 172.21.15.12:0/1774856386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55c0cbbf0 con 0x7fd55c0c7900 2023-07-08T12:38:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.366+0000 7fd5618cf700 1 -- 172.21.15.12:0/1774856386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55c04ea60 con 0x7fd55c0c7900 2023-07-08T12:38:57.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.366+0000 7fd543fff700 1 -- 172.21.15.12:0/1774856386 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd554024020 con 0x7fd55c0c7900 2023-07-08T12:38:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.367+0000 7fd543fff700 1 --2- 172.21.15.12:0/1774856386 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd54806ace0 0x7fd54806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.367+0000 7fd543fff700 1 -- 172.21.15.12:0/1774856386 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd55408bc70 con 0x7fd55c0c7900 2023-07-08T12:38:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.367+0000 7fd55affd700 1 --2- 172.21.15.12:0/1774856386 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd54806ace0 0x7fd54806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.368+0000 7fd55affd700 1 --2- 172.21.15.12:0/1774856386 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd54806ace0 0x7fd54806d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd550009c60 tx=0x7fd550009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:57.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.371+0000 7fd543fff700 1 -- 172.21.15.12:0/1774856386 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd554056b40 con 0x7fd55c0c7900 2023-07-08T12:38:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.570+0000 7fd5618cf700 1 -- 172.21.15.12:0/1774856386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd55c0ce0d0 con 0x7fd55c0c7900 2023-07-08T12:38:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.570+0000 7fd543fff700 1 -- 172.21.15.12:0/1774856386 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd55402c020 con 0x7fd55c0c7900 2023-07-08T12:38:57.573 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:38:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.572+0000 7fd5618cf700 1 -- 172.21.15.12:0/1774856386 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd54806ace0 msgr2=0x7fd54806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.572+0000 7fd5618cf700 1 --2- 172.21.15.12:0/1774856386 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd54806ace0 0x7fd54806d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd550009c60 tx=0x7fd550009210 comp rx=0 tx=0).stop 2023-07-08T12:38:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.572+0000 7fd5618cf700 1 -- 172.21.15.12:0/1774856386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd55c0c7900 msgr2=0x7fd55c0c7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.572+0000 7fd5618cf700 1 --2- 172.21.15.12:0/1774856386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd55c0c7900 0x7fd55c0c7da0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd554012000 tx=0x7fd554001d80 comp rx=0 tx=0).stop 2023-07-08T12:38:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.572+0000 7fd5618cf700 1 -- 172.21.15.12:0/1774856386 shutdown_connections 2023-07-08T12:38:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.572+0000 7fd5618cf700 1 --2- 172.21.15.12:0/1774856386 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd54806ace0 0x7fd54806d190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.572+0000 7fd5618cf700 1 --2- 172.21.15.12:0/1774856386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd55c0c7900 0x7fd55c0c7da0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.572+0000 7fd5618cf700 1 --2- 172.21.15.12:0/1774856386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd544004d90 0x7fd55c155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.572+0000 7fd5618cf700 1 --2- 172.21.15.12:0/1774856386 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd55c0ca620 0x7fd55c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.572+0000 7fd5618cf700 1 -- 172.21.15.12:0/1774856386 >> 172.21.15.12:0/1774856386 conn(0x7fd55c06b3f0 msgr2=0x7fd55c0b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.572+0000 7fd5618cf700 1 -- 172.21.15.12:0/1774856386 shutdown_connections 2023-07-08T12:38:57.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.572+0000 7fd5618cf700 1 -- 172.21.15.12:0/1774856386 wait complete. 2023-07-08T12:38:57.586 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-08T12:38:57.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.712+0000 7f0926dac700 1 Processor -- start 2023-07-08T12:38:57.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.712+0000 7f0926dac700 1 -- start start 2023-07-08T12:38:57.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.713+0000 7f0926dac700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f092006e0c0 0x7f09201608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.713+0000 7f0926dac700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0920160410 con 0x7f09200cc240 2023-07-08T12:38:57.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.713+0000 7f0926dac700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0920156de0 con 0x7f0920156a10 2023-07-08T12:38:57.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.713+0000 7f0926dac700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092006ec70 con 0x7f092006e0c0 2023-07-08T12:38:57.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.713+0000 7f0925349700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f09200cc240 0x7f09200ae440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37920/0 (socket says 172.21.15.12:37920) 2023-07-08T12:38:57.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.713+0000 7f0925349700 1 -- 172.21.15.12:0/3336615247 learned_addr learned my addr 172.21.15.12:0/3336615247 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:57.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.713+0000 7f091ffff700 1 --2- 172.21.15.12:0/3336615247 >> v2:172.21.15.160:3300/0 conn(0x7f092006e0c0 0x7f09201608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.713+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792285618 0 0) 0x7f0920160410 con 0x7f09200cc240 2023-07-08T12:38:57.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09100035b0 con 0x7f09200cc240 2023-07-08T12:38:57.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 609094166 0 0) 0x7f09100035b0 con 0x7f09200cc240 2023-07-08T12:38:57.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 >> v2:172.21.15.160:3300/0 conn(0x7f092006e0c0 msgr2=0x7f09201608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:57.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 --2- 172.21.15.12:0/3336615247 >> v2:172.21.15.160:3300/0 conn(0x7f092006e0c0 0x7f09201608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 >> v1:172.21.15.160:6789/0 conn(0x7f0920156a10 legacy=0x7f092015d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092006e6b0 con 0x7f09200cc240 2023-07-08T12:38:57.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f09180019f0 con 0x7f09200cc240 2023-07-08T12:38:57.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 >> v1:172.21.15.12:6789/0 conn(0x7f09200cc240 legacy=0x7f09200ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 --2- 172.21.15.12:0/3336615247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0910003a00 0x7f0910003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 --2- 172.21.15.12:0/3336615247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0910004850 0x7f0910004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 --2- 172.21.15.12:0/3336615247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0910005200 0x7f091000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092006ec70 con 0x7f0910004850 2023-07-08T12:38:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091000c080 con 0x7f0910003a00 2023-07-08T12:38:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091000c240 con 0x7f0910005200 2023-07-08T12:38:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091ffff700 1 --2- 172.21.15.12:0/3336615247 >> v2:172.21.15.160:3300/0 conn(0x7f092006e0c0 0x7f09201608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:38:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091ffff700 1 --2- 172.21.15.12:0/3336615247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0910004850 0x7f0910004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f0925349700 1 --2- 172.21.15.12:0/3336615247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0910005200 0x7f091000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f0924b48700 1 --2- 172.21.15.12:0/3336615247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0910003a00 0x7f0910003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091ffff700 1 -- 172.21.15.12:0/3336615247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0910005200 msgr2=0x7f091000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091ffff700 1 --2- 172.21.15.12:0/3336615247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0910005200 0x7f091000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091ffff700 1 -- 172.21.15.12:0/3336615247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0910003a00 msgr2=0x7f0910003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091ffff700 1 --2- 172.21.15.12:0/3336615247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0910003a00 0x7f0910003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f091ffff700 1 -- 172.21.15.12:0/3336615247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09100035b0 con 0x7f0910004850 2023-07-08T12:38:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.714+0000 7f0925349700 1 -- 172.21.15.12:0/3336615247 reap_dead start 2023-07-08T12:38:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.715+0000 7f091ffff700 1 --2- 172.21.15.12:0/3336615247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0910004850 0x7f0910004cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f090c009d10 tx=0x7f0910005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba794a39780dad96 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.715+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c009e60 con 0x7f0910004850 2023-07-08T12:38:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.715+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c00e9a0 con 0x7f0910004850 2023-07-08T12:38:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.715+0000 7f091f7fe700 1 -- 172.21.15.12:0/3336615247 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c00ed30 con 0x7f0910004850 2023-07-08T12:38:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.715+0000 7f0926dac700 1 -- 172.21.15.12:0/3336615247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0910004850 msgr2=0x7f0910004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.715+0000 7f0926dac700 1 --2- 172.21.15.12:0/3336615247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0910004850 0x7f0910004cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f090c009d10 tx=0x7f0910005880 comp rx=0 tx=0).stop 2023-07-08T12:38:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.715+0000 7f0926dac700 1 -- 172.21.15.12:0/3336615247 shutdown_connections 2023-07-08T12:38:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.715+0000 7f0926dac700 1 --2- 172.21.15.12:0/3336615247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0910004850 0x7f0910004cc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.715+0000 7f0926dac700 1 -- 172.21.15.12:0/3336615247 >> 172.21.15.12:0/3336615247 conn(0x7f0920098b50 msgr2=0x7f092006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.715+0000 7f0926dac700 1 -- 172.21.15.12:0/3336615247 shutdown_connections 2023-07-08T12:38:57.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.715+0000 7f0926dac700 1 -- 172.21.15.12:0/3336615247 wait complete. 2023-07-08T12:38:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.716+0000 7f0926dac700 1 Processor -- start 2023-07-08T12:38:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.716+0000 7f0926dac700 1 -- start start 2023-07-08T12:38:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.717+0000 7f0926dac700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0910004850 0x7f09200ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.717+0000 7f0926dac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f092015e370 0x7f092015e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.717+0000 7f0926dac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09200acd30 0x7f09200ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.717+0000 7f0926dac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09200ad6f0 con 0x7f09200acd30 2023-07-08T12:38:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.717+0000 7f0926dac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09200adfa0 con 0x7f092015e370 2023-07-08T12:38:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.717+0000 7f0926dac700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092015ed00 con 0x7f0910004850 2023-07-08T12:38:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.717+0000 7f0925349700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09200acd30 0x7f09200ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.717+0000 7f0925349700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09200acd30 0x7f09200ad1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50536/0 (socket says 172.21.15.12:50536) 2023-07-08T12:38:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.717+0000 7f0925349700 1 -- 172.21.15.12:0/1514948353 learned_addr learned my addr 172.21.15.12:0/1514948353 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.718+0000 7f091ffff700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f092015e370 0x7f092015e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.718+0000 7f0924b48700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0910004850 0x7f09200ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.718+0000 7f0925349700 1 -- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0910004850 msgr2=0x7f09200ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.718+0000 7f0925349700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0910004850 0x7f09200ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.718+0000 7f0925349700 1 -- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f092015e370 msgr2=0x7f092015e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.718+0000 7f0925349700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f092015e370 0x7f092015e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.718+0000 7f0925349700 1 -- 172.21.15.12:0/1514948353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09200c6240 con 0x7f09200acd30 2023-07-08T12:38:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.718+0000 7f0925349700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09200acd30 0x7f09200ad1b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f09200b19e0 tx=0x7f091800a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.718+0000 7f091d7fa700 1 -- 172.21.15.12:0/1514948353 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091800ad60 con 0x7f09200acd30 2023-07-08T12:38:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.718+0000 7f091d7fa700 1 -- 172.21.15.12:0/1514948353 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09180133f0 con 0x7f09200acd30 2023-07-08T12:38:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.718+0000 7f0926dac700 1 -- 172.21.15.12:0/1514948353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09200aede0 con 0x7f09200acd30 2023-07-08T12:38:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.718+0000 7f091d7fa700 1 -- 172.21.15.12:0/1514948353 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180137a0 con 0x7f09200acd30 2023-07-08T12:38:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.718+0000 7f0926dac700 1 -- 172.21.15.12:0/1514948353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09200ae4b0 con 0x7f09200acd30 2023-07-08T12:38:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.719+0000 7f0926dac700 1 -- 172.21.15.12:0/1514948353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09201429f0 con 0x7f09200acd30 2023-07-08T12:38:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.719+0000 7f091d7fa700 1 -- 172.21.15.12:0/1514948353 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0918013900 con 0x7f09200acd30 2023-07-08T12:38:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.720+0000 7f091d7fa700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f090406ace0 0x7f090406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.720+0000 7f091d7fa700 1 -- 172.21.15.12:0/1514948353 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f091808a200 con 0x7f09200acd30 2023-07-08T12:38:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.720+0000 7f0924b48700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f090406ace0 0x7f090406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.721+0000 7f0924b48700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f090406ace0 0x7f090406d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0914000c00 tx=0x7f0914009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:57.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.724+0000 7f091d7fa700 1 -- 172.21.15.12:0/1514948353 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f09201429f0 con 0x7f09200acd30 2023-07-08T12:38:57.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.920+0000 7f0926dac700 1 -- 172.21.15.12:0/1514948353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09201429f0 con 0x7f09200acd30 2023-07-08T12:38:57.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.920+0000 7f091d7fa700 1 -- 172.21.15.12:0/1514948353 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f09201429f0 con 0x7f09200acd30 2023-07-08T12:38:57.923 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:38:57.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.922+0000 7f0926dac700 1 -- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f090406ace0 msgr2=0x7f090406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.922+0000 7f0926dac700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f090406ace0 0x7f090406d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0914000c00 tx=0x7f0914009040 comp rx=0 tx=0).stop 2023-07-08T12:38:57.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.922+0000 7f0926dac700 1 -- 172.21.15.12:0/1514948353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09200acd30 msgr2=0x7f09200ad1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:57.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.922+0000 7f0926dac700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09200acd30 0x7f09200ad1b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f09200b19e0 tx=0x7f091800a5f0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.922+0000 7f0926dac700 1 -- 172.21.15.12:0/1514948353 shutdown_connections 2023-07-08T12:38:57.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.922+0000 7f0926dac700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f090406ace0 0x7f090406d190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.922+0000 7f0926dac700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09200acd30 0x7f09200ad1b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.922+0000 7f0926dac700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0910004850 0x7f09200ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.922+0000 7f0926dac700 1 --2- 172.21.15.12:0/1514948353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f092015e370 0x7f092015e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.922+0000 7f0926dac700 1 -- 172.21.15.12:0/1514948353 >> 172.21.15.12:0/1514948353 conn(0x7f0920098b50 msgr2=0x7f09200ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.922+0000 7f0926dac700 1 -- 172.21.15.12:0/1514948353 shutdown_connections 2023-07-08T12:38:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:57.923+0000 7f0926dac700 1 -- 172.21.15.12:0/1514948353 wait complete. 2023-07-08T12:38:57.937 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-08T12:38:58.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.063+0000 7f0261f49700 1 Processor -- start 2023-07-08T12:38:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.063+0000 7f0261f49700 1 -- start start 2023-07-08T12:38:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.064+0000 7f0261f49700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f025c0aeec0 0x7f025c06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.064+0000 7f0261f49700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c15d770 con 0x7f025c06ebe0 2023-07-08T12:38:58.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.064+0000 7f0261f49700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c15cb60 con 0x7f025c0cdb80 2023-07-08T12:38:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.064+0000 7f0261f49700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c15a600 con 0x7f025c0aeec0 2023-07-08T12:38:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.064+0000 7f025b7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f025c0aeec0 0x7f025c06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.064+0000 7f025affd700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f025c06ebe0 0x7f025c0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37936/0 (socket says 172.21.15.12:37936) 2023-07-08T12:38:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.064+0000 7f025affd700 1 -- 172.21.15.12:0/3089300167 learned_addr learned my addr 172.21.15.12:0/3089300167 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.064+0000 7f025b7fe700 1 -- 172.21.15.12:0/3089300167 >> v1:172.21.15.160:6789/0 conn(0x7f025c0cdb80 legacy=0x7f025c0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.064+0000 7f025b7fe700 1 -- 172.21.15.12:0/3089300167 >> v1:172.21.15.12:6789/0 conn(0x7f025c06ebe0 legacy=0x7f025c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.064+0000 7f025b7fe700 1 -- 172.21.15.12:0/3089300167 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c15ce40 con 0x7f025c0aeec0 2023-07-08T12:38:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.064+0000 7f025b7fe700 1 --2- 172.21.15.12:0/3089300167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f025c0aeec0 0x7f025c06cff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f024c009820 tx=0x7f024c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2956e38969d977 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.065+0000 7f025a7fc700 1 -- 172.21.15.12:0/3089300167 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c005e30 con 0x7f025c0aeec0 2023-07-08T12:38:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.065+0000 7f025a7fc700 1 -- 172.21.15.12:0/3089300167 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c006030 con 0x7f025c0aeec0 2023-07-08T12:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.065+0000 7f025a7fc700 1 -- 172.21.15.12:0/3089300167 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c0063e0 con 0x7f025c0aeec0 2023-07-08T12:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.065+0000 7f0261f49700 1 -- 172.21.15.12:0/3089300167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f025c0aeec0 msgr2=0x7f025c06cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.065+0000 7f0261f49700 1 --2- 172.21.15.12:0/3089300167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f025c0aeec0 0x7f025c06cff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f024c009820 tx=0x7f024c009b30 comp rx=0 tx=0).stop 2023-07-08T12:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.065+0000 7f0261f49700 1 -- 172.21.15.12:0/3089300167 shutdown_connections 2023-07-08T12:38:58.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.065+0000 7f0261f49700 1 --2- 172.21.15.12:0/3089300167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f025c0aeec0 0x7f025c06cff0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.065+0000 7f0261f49700 1 -- 172.21.15.12:0/3089300167 >> 172.21.15.12:0/3089300167 conn(0x7f025c098b50 msgr2=0x7f025c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:58.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.065+0000 7f0261f49700 1 -- 172.21.15.12:0/3089300167 shutdown_connections 2023-07-08T12:38:58.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.065+0000 7f0261f49700 1 -- 172.21.15.12:0/3089300167 wait complete. 2023-07-08T12:38:58.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.066+0000 7f0261f49700 1 Processor -- start 2023-07-08T12:38:58.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.067+0000 7f0261f49700 1 -- start start 2023-07-08T12:38:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.067+0000 7f0261f49700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f025c06ebe0 0x7f025c14e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.067+0000 7f0261f49700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f025c0aeec0 0x7f025c0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.067+0000 7f0261f49700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f025c0cdb80 0x7f025c1e3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.067+0000 7f0261f49700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0b15f0 con 0x7f025c0aeec0 2023-07-08T12:38:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.067+0000 7f0261f49700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0b0c20 con 0x7f025c06ebe0 2023-07-08T12:38:58.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.067+0000 7f0261f49700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0b5730 con 0x7f025c0cdb80 2023-07-08T12:38:58.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.067+0000 7f025affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f025c0aeec0 0x7f025c0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.067+0000 7f025affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f025c0aeec0 0x7f025c0b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50548/0 (socket says 172.21.15.12:50548) 2023-07-08T12:38:58.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f025affd700 1 -- 172.21.15.12:0/4104359472 learned_addr learned my addr 172.21.15.12:0/4104359472 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:58.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f025b7fe700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f025c06ebe0 0x7f025c14e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f025bfff700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f025c0cdb80 0x7f025c1e3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f025affd700 1 -- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f025c0cdb80 msgr2=0x7f025c1e3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f025affd700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f025c0cdb80 0x7f025c1e3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f025affd700 1 -- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f025c06ebe0 msgr2=0x7f025c14e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f025affd700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f025c06ebe0 0x7f025c14e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f025affd700 1 -- 172.21.15.12:0/4104359472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c009430 con 0x7f025c0aeec0 2023-07-08T12:38:58.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f025b7fe700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f025c06ebe0 0x7f025c14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:58.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f025affd700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f025c0aeec0 0x7f025c0b10b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f024400c520 tx=0x7f024400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:58.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f0253fff700 1 -- 172.21.15.12:0/4104359472 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244005d70 con 0x7f025c0aeec0 2023-07-08T12:38:58.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f0253fff700 1 -- 172.21.15.12:0/4104359472 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02440070d0 con 0x7f025c0aeec0 2023-07-08T12:38:58.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f0261f49700 1 -- 172.21.15.12:0/4104359472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c0b0010 con 0x7f025c0aeec0 2023-07-08T12:38:58.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f0253fff700 1 -- 172.21.15.12:0/4104359472 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244007440 con 0x7f025c0aeec0 2023-07-08T12:38:58.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.068+0000 7f0261f49700 1 -- 172.21.15.12:0/4104359472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c0b05d0 con 0x7f025c0aeec0 2023-07-08T12:38:58.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.069+0000 7f0261f49700 1 -- 172.21.15.12:0/4104359472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c04ea60 con 0x7f025c0aeec0 2023-07-08T12:38:58.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.070+0000 7f0253fff700 1 -- 172.21.15.12:0/4104359472 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f02440075a0 con 0x7f025c0aeec0 2023-07-08T12:38:58.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.070+0000 7f0253fff700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f023c06ace0 0x7f023c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.070+0000 7f0253fff700 1 -- 172.21.15.12:0/4104359472 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0244088bc0 con 0x7f025c0aeec0 2023-07-08T12:38:58.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.070+0000 7f025b7fe700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f023c06ace0 0x7f023c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.071+0000 7f025b7fe700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f023c06ace0 0x7f023c06d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f024c013040 tx=0x7f024c008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:58.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.074+0000 7f0253fff700 1 -- 172.21.15.12:0/4104359472 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f02440539f0 con 0x7f025c0aeec0 2023-07-08T12:38:58.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.272+0000 7f0261f49700 1 -- 172.21.15.12:0/4104359472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f025c0b0d60 con 0x7f025c0aeec0 2023-07-08T12:38:58.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.272+0000 7f0253fff700 1 -- 172.21.15.12:0/4104359472 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0244057010 con 0x7f025c0aeec0 2023-07-08T12:38:58.275 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-07-08T12:38:58.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.274+0000 7f0261f49700 1 -- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f023c06ace0 msgr2=0x7f023c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.274+0000 7f0261f49700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f023c06ace0 0x7f023c06d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f024c013040 tx=0x7f024c008070 comp rx=0 tx=0).stop 2023-07-08T12:38:58.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.274+0000 7f0261f49700 1 -- 172.21.15.12:0/4104359472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f025c0aeec0 msgr2=0x7f025c0b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.274+0000 7f0261f49700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f025c0aeec0 0x7f025c0b10b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f024400c520 tx=0x7f024400c8e0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.274+0000 7f0261f49700 1 -- 172.21.15.12:0/4104359472 shutdown_connections 2023-07-08T12:38:58.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.274+0000 7f0261f49700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f023c06ace0 0x7f023c06d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.274+0000 7f0261f49700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f025c0cdb80 0x7f025c1e3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.274+0000 7f0261f49700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f025c06ebe0 0x7f025c14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.274+0000 7f0261f49700 1 --2- 172.21.15.12:0/4104359472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f025c0aeec0 0x7f025c0b10b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.274+0000 7f0261f49700 1 -- 172.21.15.12:0/4104359472 >> 172.21.15.12:0/4104359472 conn(0x7f025c098b50 msgr2=0x7f025c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:58.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.274+0000 7f0261f49700 1 -- 172.21.15.12:0/4104359472 shutdown_connections 2023-07-08T12:38:58.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.274+0000 7f0261f49700 1 -- 172.21.15.12:0/4104359472 wait complete. 2023-07-08T12:38:58.288 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-08T12:38:58.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.415+0000 7fadaa9e0700 1 Processor -- start 2023-07-08T12:38:58.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.415+0000 7fadaa9e0700 1 -- start start 2023-07-08T12:38:58.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fadaa9e0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fada40aeec0 0x7fada406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fadaa9e0700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fada406ebe0 0x7fada40cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fadaa9e0700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fada40cd9b0 0x7fada40cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fadaa9e0700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada415d770 con 0x7fada40aeec0 2023-07-08T12:38:58.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fadaa9e0700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada40cddc0 con 0x7fada406ebe0 2023-07-08T12:38:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fadaa9e0700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada415cb60 con 0x7fada40cd9b0 2023-07-08T12:38:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fada3fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fada40aeec0 0x7fada406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fada3fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fada40aeec0 0x7fada406cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50554/0 (socket says 172.21.15.12:50554) 2023-07-08T12:38:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fada3fff700 1 -- 172.21.15.12:0/3659639603 learned_addr learned my addr 172.21.15.12:0/3659639603 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fada37fe700 1 --2- 172.21.15.12:0/3659639603 >> v2:172.21.15.160:3300/0 conn(0x7fada406ebe0 0x7fada40cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fada8f7d700 1 --2- 172.21.15.12:0/3659639603 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fada40cd9b0 0x7fada40cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fada3fff700 1 -- 172.21.15.12:0/3659639603 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fada40cd9b0 msgr2=0x7fada40cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:58.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fada3fff700 1 --2- 172.21.15.12:0/3659639603 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fada40cd9b0 0x7fada40cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fada3fff700 1 -- 172.21.15.12:0/3659639603 >> v2:172.21.15.160:3300/0 conn(0x7fada406ebe0 msgr2=0x7fada40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:58.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fada3fff700 1 --2- 172.21.15.12:0/3659639603 >> v2:172.21.15.160:3300/0 conn(0x7fada406ebe0 0x7fada40cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.416+0000 7fada3fff700 1 -- 172.21.15.12:0/3659639603 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada415ce40 con 0x7fada40aeec0 2023-07-08T12:38:58.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.417+0000 7fada3fff700 1 --2- 172.21.15.12:0/3659639603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada40aeec0 0x7fada406cff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fada406e430 tx=0x7fad9400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2f26dbc6863b58 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:58.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.418+0000 7fada2ffd700 1 -- 172.21.15.12:0/3659639603 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94005e30 con 0x7fada40aeec0 2023-07-08T12:38:58.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.418+0000 7fada2ffd700 1 -- 172.21.15.12:0/3659639603 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad94004030 con 0x7fada40aeec0 2023-07-08T12:38:58.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.418+0000 7fada2ffd700 1 -- 172.21.15.12:0/3659639603 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad940043e0 con 0x7fada40aeec0 2023-07-08T12:38:58.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.419+0000 7fadaa9e0700 1 -- 172.21.15.12:0/3659639603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada40aeec0 msgr2=0x7fada406cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.419+0000 7fadaa9e0700 1 --2- 172.21.15.12:0/3659639603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada40aeec0 0x7fada406cff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fada406e430 tx=0x7fad9400be70 comp rx=0 tx=0).stop 2023-07-08T12:38:58.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.419+0000 7fadaa9e0700 1 -- 172.21.15.12:0/3659639603 shutdown_connections 2023-07-08T12:38:58.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.419+0000 7fadaa9e0700 1 --2- 172.21.15.12:0/3659639603 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fada40cd9b0 0x7fada40cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.419+0000 7fadaa9e0700 1 --2- 172.21.15.12:0/3659639603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada40aeec0 0x7fada406cff0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.419+0000 7fadaa9e0700 1 --2- 172.21.15.12:0/3659639603 >> v2:172.21.15.160:3300/0 conn(0x7fada406ebe0 0x7fada40cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.419+0000 7fadaa9e0700 1 -- 172.21.15.12:0/3659639603 >> 172.21.15.12:0/3659639603 conn(0x7fada4098b50 msgr2=0x7fada406b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:58.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.419+0000 7fadaa9e0700 1 -- 172.21.15.12:0/3659639603 shutdown_connections 2023-07-08T12:38:58.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.419+0000 7fadaa9e0700 1 -- 172.21.15.12:0/3659639603 wait complete. 2023-07-08T12:38:58.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.421+0000 7fadaa9e0700 1 Processor -- start 2023-07-08T12:38:58.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.421+0000 7fadaa9e0700 1 -- start start 2023-07-08T12:38:58.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fadaa9e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada406ebe0 0x7fada40b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fadaa9e0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fada40aeec0 0x7fada4149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fadaa9e0700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fada40cd9b0 0x7fada40b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fadaa9e0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada40b5580 con 0x7fada406ebe0 2023-07-08T12:38:58.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fadaa9e0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada40b2cf0 con 0x7fada40aeec0 2023-07-08T12:38:58.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fadaa9e0700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada40b2fd0 con 0x7fada40cd9b0 2023-07-08T12:38:58.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada406ebe0 0x7fada40b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada406ebe0 0x7fada40b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50560/0 (socket says 172.21.15.12:50560) 2023-07-08T12:38:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada3fff700 1 -- 172.21.15.12:0/18121876 learned_addr learned my addr 172.21.15.12:0/18121876 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada37fe700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fada40aeec0 0x7fada4149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada3fff700 1 -- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fada40cd9b0 msgr2=0x7fada40b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada8f7d700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fada40cd9b0 0x7fada40b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada3fff700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fada40cd9b0 0x7fada40b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada3fff700 1 -- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fada40aeec0 msgr2=0x7fada4149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada3fff700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fada40aeec0 0x7fada4149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada3fff700 1 -- 172.21.15.12:0/18121876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9400b7a0 con 0x7fada406ebe0 2023-07-08T12:38:58.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada37fe700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fada40aeec0 0x7fada4149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:58.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada8f7d700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fada40cd9b0 0x7fada40b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:58.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada3fff700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada406ebe0 0x7fada40b2660 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fad9400b340 tx=0x7fad94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:58.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada0ff9700 1 -- 172.21.15.12:0/18121876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94003ab0 con 0x7fada406ebe0 2023-07-08T12:38:58.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fada0ff9700 1 -- 172.21.15.12:0/18121876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad94003c30 con 0x7fada406ebe0 2023-07-08T12:38:58.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.422+0000 7fadaa9e0700 1 -- 172.21.15.12:0/18121876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada40b32b0 con 0x7fada406ebe0 2023-07-08T12:38:58.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.423+0000 7fada0ff9700 1 -- 172.21.15.12:0/18121876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94004590 con 0x7fada406ebe0 2023-07-08T12:38:58.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.423+0000 7fadaa9e0700 1 -- 172.21.15.12:0/18121876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada40b3870 con 0x7fada406ebe0 2023-07-08T12:38:58.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.424+0000 7fadaa9e0700 1 -- 172.21.15.12:0/18121876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada40b3a60 con 0x7fada406ebe0 2023-07-08T12:38:58.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.424+0000 7fada0ff9700 1 -- 172.21.15.12:0/18121876 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad940193f0 con 0x7fada406ebe0 2023-07-08T12:38:58.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.424+0000 7fada0ff9700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fad9006ace0 0x7fad9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.424+0000 7fada0ff9700 1 -- 172.21.15.12:0/18121876 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fad9408ccf0 con 0x7fada406ebe0 2023-07-08T12:38:58.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.424+0000 7fada37fe700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fad9006ace0 0x7fad9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.425+0000 7fada37fe700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fad9006ace0 0x7fad9006d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fada4159470 tx=0x7fad98008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:58.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.429+0000 7fada0ff9700 1 -- 172.21.15.12:0/18121876 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fad94057b00 con 0x7fada406ebe0 2023-07-08T12:38:58.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.632+0000 7fadaa9e0700 1 -- 172.21.15.12:0/18121876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fada4066af0 con 0x7fada406ebe0 2023-07-08T12:38:58.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.632+0000 7fada0ff9700 1 -- 172.21.15.12:0/18121876 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fad9405b120 con 0x7fada406ebe0 2023-07-08T12:38:58.635 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:38:58.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.634+0000 7fadaa9e0700 1 -- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fad9006ace0 msgr2=0x7fad9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.634+0000 7fadaa9e0700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fad9006ace0 0x7fad9006d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fada4159470 tx=0x7fad98008040 comp rx=0 tx=0).stop 2023-07-08T12:38:58.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.634+0000 7fadaa9e0700 1 -- 172.21.15.12:0/18121876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada406ebe0 msgr2=0x7fada40b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.634+0000 7fadaa9e0700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada406ebe0 0x7fada40b2660 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fad9400b340 tx=0x7fad94004750 comp rx=0 tx=0).stop 2023-07-08T12:38:58.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.634+0000 7fadaa9e0700 1 -- 172.21.15.12:0/18121876 shutdown_connections 2023-07-08T12:38:58.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.634+0000 7fadaa9e0700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fad9006ace0 0x7fad9006d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.634+0000 7fadaa9e0700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fada40cd9b0 0x7fada40b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.634+0000 7fadaa9e0700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada406ebe0 0x7fada40b2660 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.634+0000 7fadaa9e0700 1 --2- 172.21.15.12:0/18121876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fada40aeec0 0x7fada4149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.634+0000 7fadaa9e0700 1 -- 172.21.15.12:0/18121876 >> 172.21.15.12:0/18121876 conn(0x7fada4098b50 msgr2=0x7fada40ac420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:58.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.634+0000 7fadaa9e0700 1 -- 172.21.15.12:0/18121876 shutdown_connections 2023-07-08T12:38:58.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.634+0000 7fadaa9e0700 1 -- 172.21.15.12:0/18121876 wait complete. 2023-07-08T12:38:58.649 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:38:58.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.771+0000 7f468601f700 1 Processor -- start 2023-07-08T12:38:58.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.771+0000 7f468601f700 1 -- start start 2023-07-08T12:38:58.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.772+0000 7f468601f700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f468006a640 0x7f468006ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.772+0000 7f468601f700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468015a770 con 0x7f468015e990 2023-07-08T12:38:58.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.772+0000 7f468601f700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468015e500 con 0x7f468015a3a0 2023-07-08T12:38:58.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.772+0000 7f468601f700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46800ca570 con 0x7f468006a640 2023-07-08T12:38:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.772+0000 7f467effd700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f468015e990 0x7f4680069f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37966/0 (socket says 172.21.15.12:37966) 2023-07-08T12:38:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.772+0000 7f467effd700 1 -- 172.21.15.12:0/643198771 learned_addr learned my addr 172.21.15.12:0/643198771 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:38:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.772+0000 7f467ffff700 1 --2- 172.21.15.12:0/643198771 >> v2:172.21.15.160:3300/0 conn(0x7f468006a640 0x7f468006ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.773+0000 7f467e7fc700 1 -- 172.21.15.12:0/643198771 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529844590 0 0) 0x7f4668001110 con 0x7f468015e990 2023-07-08T12:38:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.773+0000 7f467e7fc700 1 -- 172.21.15.12:0/643198771 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f466c0035b0 con 0x7f468015e990 2023-07-08T12:38:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.773+0000 7f467e7fc700 1 -- 172.21.15.12:0/643198771 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718441393 0 0) 0x7f468015e500 con 0x7f468015a3a0 2023-07-08T12:38:58.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.773+0000 7f467e7fc700 1 -- 172.21.15.12:0/643198771 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4668001110 con 0x7f468015a3a0 2023-07-08T12:38:58.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.773+0000 7f467ffff700 1 -- 172.21.15.12:0/643198771 >> v1:172.21.15.160:6789/0 conn(0x7f468015a3a0 legacy=0x7f46800c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.773+0000 7f467ffff700 1 -- 172.21.15.12:0/643198771 >> v1:172.21.15.12:6789/0 conn(0x7f468015e990 legacy=0x7f4680069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.773+0000 7f467ffff700 1 -- 172.21.15.12:0/643198771 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800cbd10 con 0x7f468006a640 2023-07-08T12:38:58.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.773+0000 7f467ffff700 1 --2- 172.21.15.12:0/643198771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468006a640 0x7f468006ef10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4678009d90 tx=0x7f467800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69b54fe2d64bb3a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:58.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.773+0000 7f467e7fc700 1 -- 172.21.15.12:0/643198771 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467800e070 con 0x7f468006a640 2023-07-08T12:38:58.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.773+0000 7f467e7fc700 1 -- 172.21.15.12:0/643198771 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467800bb20 con 0x7f468006a640 2023-07-08T12:38:58.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.773+0000 7f467e7fc700 1 -- 172.21.15.12:0/643198771 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467800be90 con 0x7f468006a640 2023-07-08T12:38:58.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.774+0000 7f468601f700 1 -- 172.21.15.12:0/643198771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468006a640 msgr2=0x7f468006ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.774+0000 7f468601f700 1 --2- 172.21.15.12:0/643198771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468006a640 0x7f468006ef10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4678009d90 tx=0x7f467800b330 comp rx=0 tx=0).stop 2023-07-08T12:38:58.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.774+0000 7f468601f700 1 -- 172.21.15.12:0/643198771 shutdown_connections 2023-07-08T12:38:58.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.774+0000 7f468601f700 1 --2- 172.21.15.12:0/643198771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468006a640 0x7f468006ef10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.774+0000 7f468601f700 1 -- 172.21.15.12:0/643198771 >> 172.21.15.12:0/643198771 conn(0x7f46800cc290 msgr2=0x7f46800cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:58.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.774+0000 7f468601f700 1 -- 172.21.15.12:0/643198771 shutdown_connections 2023-07-08T12:38:58.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.774+0000 7f468601f700 1 -- 172.21.15.12:0/643198771 wait complete. 2023-07-08T12:38:58.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.775+0000 7f468601f700 1 Processor -- start 2023-07-08T12:38:58.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.775+0000 7f468601f700 1 -- start start 2023-07-08T12:38:58.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f468601f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468006a640 0x7f4680158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f468601f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a3a0 0x7f46801e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f468601f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f468015e990 0x7f4680157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f468601f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46801e5a60 con 0x7f468015e990 2023-07-08T12:38:58.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f468601f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468015b320 con 0x7f468006a640 2023-07-08T12:38:58.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f468601f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468015b600 con 0x7f468015a3a0 2023-07-08T12:38:58.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f467ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f468015e990 0x7f4680157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f467ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f468015e990 0x7f4680157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50578/0 (socket says 172.21.15.12:50578) 2023-07-08T12:38:58.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f467ffff700 1 -- 172.21.15.12:0/3809711608 learned_addr learned my addr 172.21.15.12:0/3809711608 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:58.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f467f7fe700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468006a640 0x7f4680158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f467effd700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a3a0 0x7f46801e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f467ffff700 1 -- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a3a0 msgr2=0x7f46801e53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f467ffff700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a3a0 0x7f46801e53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f467ffff700 1 -- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468006a640 msgr2=0x7f4680158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f467ffff700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468006a640 0x7f4680158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f467ffff700 1 -- 172.21.15.12:0/3809711608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4678009a70 con 0x7f468015e990 2023-07-08T12:38:58.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.776+0000 7f467effd700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a3a0 0x7f46801e53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:58.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.777+0000 7f467ffff700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f468015e990 0x7f4680157f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4678013000 tx=0x7f4678004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:58.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.777+0000 7f4677fff700 1 -- 172.21.15.12:0/3809711608 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46780098f0 con 0x7f468015e990 2023-07-08T12:38:58.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.777+0000 7f4677fff700 1 -- 172.21.15.12:0/3809711608 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4678020070 con 0x7f468015e990 2023-07-08T12:38:58.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.777+0000 7f468601f700 1 -- 172.21.15.12:0/3809711608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468015b8e0 con 0x7f468015e990 2023-07-08T12:38:58.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.777+0000 7f4677fff700 1 -- 172.21.15.12:0/3809711608 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46780276c0 con 0x7f468015e990 2023-07-08T12:38:58.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.777+0000 7f468601f700 1 -- 172.21.15.12:0/3809711608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4680158f70 con 0x7f468015e990 2023-07-08T12:38:58.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.778+0000 7f468601f700 1 -- 172.21.15.12:0/3809711608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468004ea60 con 0x7f468015e990 2023-07-08T12:38:58.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.779+0000 7f4677fff700 1 -- 172.21.15.12:0/3809711608 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4678031430 con 0x7f468015e990 2023-07-08T12:38:58.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.779+0000 7f4677fff700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f466006ace0 0x7f466006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:58.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.780+0000 7f4677fff700 1 -- 172.21.15.12:0/3809711608 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4678022070 con 0x7f468015e990 2023-07-08T12:38:58.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.780+0000 7f467f7fe700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f466006ace0 0x7f466006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:58.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.780+0000 7f467f7fe700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f466006ace0 0x7f466006d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f46801e5300 tx=0x7f4670009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:58.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.783+0000 7f4677fff700 1 -- 172.21.15.12:0/3809711608 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4678056930 con 0x7f468015e990 2023-07-08T12:38:58.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.982+0000 7f468601f700 1 -- 172.21.15.12:0/3809711608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f468006adc0 con 0x7f468015e990 2023-07-08T12:38:58.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.983+0000 7f4677fff700 1 -- 172.21.15.12:0/3809711608 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20203 (secure 0 0 0) 0x7f4678059f50 con 0x7f468015e990 2023-07-08T12:38:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:38:58.986 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:49.386462+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:38:58.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.985+0000 7f468601f700 1 -- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f466006ace0 msgr2=0x7f466006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.985+0000 7f468601f700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f466006ace0 0x7f466006d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f46801e5300 tx=0x7f4670009040 comp rx=0 tx=0).stop 2023-07-08T12:38:58.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.985+0000 7f468601f700 1 -- 172.21.15.12:0/3809711608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f468015e990 msgr2=0x7f4680157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:58.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.985+0000 7f468601f700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f468015e990 0x7f4680157f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4678013000 tx=0x7f4678004ad0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.985+0000 7f468601f700 1 -- 172.21.15.12:0/3809711608 shutdown_connections 2023-07-08T12:38:58.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.985+0000 7f468601f700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f466006ace0 0x7f466006d190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.985+0000 7f468601f700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f468015e990 0x7f4680157f30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.985+0000 7f468601f700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468006a640 0x7f4680158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.985+0000 7f468601f700 1 --2- 172.21.15.12:0/3809711608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a3a0 0x7f46801e53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:58.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.985+0000 7f468601f700 1 -- 172.21.15.12:0/3809711608 >> 172.21.15.12:0/3809711608 conn(0x7f46800cc290 msgr2=0x7f46800c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:58.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.985+0000 7f468601f700 1 -- 172.21.15.12:0/3809711608 shutdown_connections 2023-07-08T12:38:58.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:58.985+0000 7f468601f700 1 -- 172.21.15.12:0/3809711608 wait complete. 2023-07-08T12:38:59.000 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-08T12:38:05.660194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-08T12:38:06.920177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-08T12:38:10.592749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-08T12:38:59.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:38:59.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.127+0000 7f211120b700 1 Processor -- start 2023-07-08T12:38:59.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.128+0000 7f211120b700 1 -- start start 2023-07-08T12:38:59.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.128+0000 7f211120b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f210c156a10 0x7f210c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.128+0000 7f211120b700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f210c0cc240 0x7f210c0cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.128+0000 7f211120b700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210c0ccbe0 con 0x7f210c15d390 2023-07-08T12:38:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.128+0000 7f211120b700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210c160410 con 0x7f210c156a10 2023-07-08T12:38:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.128+0000 7f211120b700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210c156de0 con 0x7f210c0cc240 2023-07-08T12:38:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.128+0000 7f210ad9d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f210c156a10 0x7f210c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.129+0000 7f210ad9d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f210c156a10 0x7f210c06e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50582/0 (socket says 172.21.15.12:50582) 2023-07-08T12:38:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.129+0000 7f210ad9d700 1 -- 172.21.15.12:0/336550871 learned_addr learned my addr 172.21.15.12:0/336550871 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.129+0000 7f210b59e700 1 --2- 172.21.15.12:0/336550871 >> v2:172.21.15.160:3300/0 conn(0x7f210c0cc240 0x7f210c0cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.129+0000 7f210ad9d700 1 -- 172.21.15.12:0/336550871 >> v2:172.21.15.160:3300/0 conn(0x7f210c0cc240 msgr2=0x7f210c0cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.129+0000 7f210ad9d700 1 --2- 172.21.15.12:0/336550871 >> v2:172.21.15.160:3300/0 conn(0x7f210c0cc240 0x7f210c0cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.129+0000 7f210ad9d700 1 -- 172.21.15.12:0/336550871 >> v1:172.21.15.160:6789/0 conn(0x7f210c15d390 legacy=0x7f210c1608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.129+0000 7f210ad9d700 1 -- 172.21.15.12:0/336550871 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c06ec70 con 0x7f210c156a10 2023-07-08T12:38:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.129+0000 7f210ad9d700 1 --2- 172.21.15.12:0/336550871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f210c156a10 0x7f210c06e0c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f210c06c880 tx=0x7f20fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a2d49f9fdfbc75 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.129+0000 7f2109d9b700 1 -- 172.21.15.12:0/336550871 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc005e30 con 0x7f210c156a10 2023-07-08T12:38:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.129+0000 7f2109d9b700 1 -- 172.21.15.12:0/336550871 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc004030 con 0x7f210c156a10 2023-07-08T12:38:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.129+0000 7f2109d9b700 1 -- 172.21.15.12:0/336550871 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0043e0 con 0x7f210c156a10 2023-07-08T12:38:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.130+0000 7f211120b700 1 -- 172.21.15.12:0/336550871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f210c156a10 msgr2=0x7f210c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.130+0000 7f211120b700 1 --2- 172.21.15.12:0/336550871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f210c156a10 0x7f210c06e0c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f210c06c880 tx=0x7f20fc009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.130+0000 7f211120b700 1 -- 172.21.15.12:0/336550871 shutdown_connections 2023-07-08T12:38:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.130+0000 7f211120b700 1 --2- 172.21.15.12:0/336550871 >> v2:172.21.15.160:3300/0 conn(0x7f210c0cc240 0x7f210c0cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.130+0000 7f211120b700 1 --2- 172.21.15.12:0/336550871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f210c156a10 0x7f210c06e0c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.130+0000 7f211120b700 1 -- 172.21.15.12:0/336550871 >> 172.21.15.12:0/336550871 conn(0x7f210c098b50 msgr2=0x7f210c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.130+0000 7f211120b700 1 -- 172.21.15.12:0/336550871 shutdown_connections 2023-07-08T12:38:59.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.130+0000 7f211120b700 1 -- 172.21.15.12:0/336550871 wait complete. 2023-07-08T12:38:59.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.131+0000 7f211120b700 1 Processor -- start 2023-07-08T12:38:59.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.131+0000 7f211120b700 1 -- start start 2023-07-08T12:38:59.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f211120b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f210c0cc240 0x7f210c1e7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f211120b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f210c156a10 0x7f210c0ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f211120b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f210c15d390 0x7f210c154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f211120b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c1e8030 con 0x7f210c0cc240 2023-07-08T12:38:59.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f211120b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c1e7660 con 0x7f210c15d390 2023-07-08T12:38:59.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f211120b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c1e6a50 con 0x7f210c156a10 2023-07-08T12:38:59.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f210ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f210c0cc240 0x7f210c1e7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f210ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f210c0cc240 0x7f210c1e7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50588/0 (socket says 172.21.15.12:50588) 2023-07-08T12:38:59.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f210ad9d700 1 -- 172.21.15.12:0/3733399373 learned_addr learned my addr 172.21.15.12:0/3733399373 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:59.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f210b59e700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f210c15d390 0x7f210c154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f210a59c700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f210c156a10 0x7f210c0ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f210ad9d700 1 -- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f210c156a10 msgr2=0x7f210c0ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f210ad9d700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f210c156a10 0x7f210c0ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f210ad9d700 1 -- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f210c15d390 msgr2=0x7f210c154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.132+0000 7f210ad9d700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f210c15d390 0x7f210c154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.133+0000 7f210ad9d700 1 -- 172.21.15.12:0/3733399373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc009500 con 0x7f210c0cc240 2023-07-08T12:38:59.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.133+0000 7f210ad9d700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f210c0cc240 0x7f210c1e7af0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009c10 tx=0x7f20fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:59.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.133+0000 7f20f37fe700 1 -- 172.21.15.12:0/3733399373 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc003c30 con 0x7f210c0cc240 2023-07-08T12:38:59.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.133+0000 7f20f37fe700 1 -- 172.21.15.12:0/3733399373 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc003d90 con 0x7f210c0cc240 2023-07-08T12:38:59.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.133+0000 7f211120b700 1 -- 172.21.15.12:0/3733399373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c1e6d30 con 0x7f210c0cc240 2023-07-08T12:38:59.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.133+0000 7f20f37fe700 1 -- 172.21.15.12:0/3733399373 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc01c640 con 0x7f210c0cc240 2023-07-08T12:38:59.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.133+0000 7f211120b700 1 -- 172.21.15.12:0/3733399373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c1e72f0 con 0x7f210c0cc240 2023-07-08T12:38:59.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.134+0000 7f211120b700 1 -- 172.21.15.12:0/3733399373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c1e7480 con 0x7f210c0cc240 2023-07-08T12:38:59.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.134+0000 7f20f37fe700 1 -- 172.21.15.12:0/3733399373 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f20fc01c7a0 con 0x7f210c0cc240 2023-07-08T12:38:59.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.135+0000 7f20f37fe700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f20f806ace0 0x7f20f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.135+0000 7f20f37fe700 1 -- 172.21.15.12:0/3733399373 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f20fc08a760 con 0x7f210c0cc240 2023-07-08T12:38:59.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.135+0000 7f210a59c700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f20f806ace0 0x7f20f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.135+0000 7f210a59c700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f20f806ace0 0x7f20f806d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f210c0b4da0 tx=0x7f2100009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:59.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.139+0000 7f20f37fe700 1 -- 172.21.15.12:0/3733399373 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f20fc0554d0 con 0x7f210c0cc240 2023-07-08T12:38:59.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.337+0000 7f211120b700 1 -- 172.21.15.12:0/3733399373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f210c1e7940 con 0x7f210c0cc240 2023-07-08T12:38:59.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.338+0000 7f20f37fe700 1 -- 172.21.15.12:0/3733399373 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20fc058af0 con 0x7f210c0cc240 2023-07-08T12:38:59.341 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:38:59.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.340+0000 7f211120b700 1 -- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f20f806ace0 msgr2=0x7f20f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.340+0000 7f211120b700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f20f806ace0 0x7f20f806d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f210c0b4da0 tx=0x7f2100009040 comp rx=0 tx=0).stop 2023-07-08T12:38:59.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.340+0000 7f211120b700 1 -- 172.21.15.12:0/3733399373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f210c0cc240 msgr2=0x7f210c1e7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.340+0000 7f211120b700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f210c0cc240 0x7f210c1e7af0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009c10 tx=0x7f20fc004750 comp rx=0 tx=0).stop 2023-07-08T12:38:59.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.340+0000 7f211120b700 1 -- 172.21.15.12:0/3733399373 shutdown_connections 2023-07-08T12:38:59.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.340+0000 7f211120b700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f20f806ace0 0x7f20f806d190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.340+0000 7f211120b700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f210c15d390 0x7f210c154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.340+0000 7f211120b700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f210c0cc240 0x7f210c1e7af0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.340+0000 7f211120b700 1 --2- 172.21.15.12:0/3733399373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f210c156a10 0x7f210c0ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.340+0000 7f211120b700 1 -- 172.21.15.12:0/3733399373 >> 172.21.15.12:0/3733399373 conn(0x7f210c098b50 msgr2=0x7f210c15f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:59.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.340+0000 7f211120b700 1 -- 172.21.15.12:0/3733399373 shutdown_connections 2023-07-08T12:38:59.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.340+0000 7f211120b700 1 -- 172.21.15.12:0/3733399373 wait complete. 2023-07-08T12:38:59.354 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:38:59.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.480+0000 7f3504f96700 1 Processor -- start 2023-07-08T12:38:59.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.480+0000 7f3504f96700 1 -- start start 2023-07-08T12:38:59.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.481+0000 7f3504f96700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f350015a190 0x7f350006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.481+0000 7f3504f96700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f35000acb30 0x7f3500163c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.481+0000 7f3504f96700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35000acf70 con 0x7f35000b12b0 2023-07-08T12:38:59.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.481+0000 7f3504f96700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350015a560 con 0x7f35000acb30 2023-07-08T12:38:59.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.481+0000 7f3504f96700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350006d2b0 con 0x7f350015a190 2023-07-08T12:38:59.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.481+0000 7f34fed9d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f35000acb30 0x7f3500163c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fed9d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f35000acb30 0x7f3500163c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50596/0 (socket says 172.21.15.12:50596) 2023-07-08T12:38:59.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fdd9b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f35000b12b0 0x7f35000ac420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37990/0 (socket says 172.21.15.12:37990) 2023-07-08T12:38:59.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fed9d700 1 -- 172.21.15.12:0/4094079799 learned_addr learned my addr 172.21.15.12:0/4094079799 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:59.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fe59c700 1 --2- 172.21.15.12:0/4094079799 >> v2:172.21.15.160:3300/0 conn(0x7f350015a190 0x7f350006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fed9d700 1 -- 172.21.15.12:0/4094079799 >> v2:172.21.15.160:3300/0 conn(0x7f350015a190 msgr2=0x7f350006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:59.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fed9d700 1 --2- 172.21.15.12:0/4094079799 >> v2:172.21.15.160:3300/0 conn(0x7f350015a190 0x7f350006e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fed9d700 1 -- 172.21.15.12:0/4094079799 >> v1:172.21.15.12:6789/0 conn(0x7f35000b12b0 legacy=0x7f35000ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fed9d700 1 -- 172.21.15.12:0/4094079799 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350006d590 con 0x7f35000acb30 2023-07-08T12:38:59.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fd59a700 1 -- 172.21.15.12:0/4094079799 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568380434 0 0) 0x7f35000acf70 con 0x7f35000b12b0 2023-07-08T12:38:59.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fe59c700 1 --2- 172.21.15.12:0/4094079799 >> v2:172.21.15.160:3300/0 conn(0x7f350015a190 0x7f350006e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:59.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fed9d700 1 --2- 172.21.15.12:0/4094079799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35000acb30 0x7f3500163c90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f350006c0d0 tx=0x7f34f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80437287971cc0c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:59.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fd59a700 1 -- 172.21.15.12:0/4094079799 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f800e070 con 0x7f35000acb30 2023-07-08T12:38:59.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fd59a700 1 -- 172.21.15.12:0/4094079799 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f800b9c0 con 0x7f35000acb30 2023-07-08T12:38:59.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.482+0000 7f34fd59a700 1 -- 172.21.15.12:0/4094079799 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f800bd70 con 0x7f35000acb30 2023-07-08T12:38:59.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.483+0000 7f3504f96700 1 -- 172.21.15.12:0/4094079799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35000acb30 msgr2=0x7f3500163c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.483+0000 7f3504f96700 1 --2- 172.21.15.12:0/4094079799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35000acb30 0x7f3500163c90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f350006c0d0 tx=0x7f34f800b330 comp rx=0 tx=0).stop 2023-07-08T12:38:59.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.483+0000 7f3504f96700 1 -- 172.21.15.12:0/4094079799 shutdown_connections 2023-07-08T12:38:59.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.483+0000 7f3504f96700 1 --2- 172.21.15.12:0/4094079799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35000acb30 0x7f3500163c90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.483+0000 7f3504f96700 1 --2- 172.21.15.12:0/4094079799 >> v2:172.21.15.160:3300/0 conn(0x7f350015a190 0x7f350006e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.483+0000 7f3504f96700 1 -- 172.21.15.12:0/4094079799 >> 172.21.15.12:0/4094079799 conn(0x7f3500098b50 msgr2=0x7f350006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:59.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.483+0000 7f3504f96700 1 -- 172.21.15.12:0/4094079799 shutdown_connections 2023-07-08T12:38:59.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.483+0000 7f3504f96700 1 -- 172.21.15.12:0/4094079799 wait complete. 2023-07-08T12:38:59.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.484+0000 7f3504f96700 1 Processor -- start 2023-07-08T12:38:59.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.484+0000 7f3504f96700 1 -- start start 2023-07-08T12:38:59.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f3504f96700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35000acb30 0x7f3500154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f3504f96700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35000b12b0 0x7f35000b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f3504f96700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f350015a190 0x7f35000b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f3504f96700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35000c3ec0 con 0x7f35000b12b0 2023-07-08T12:38:59.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f3504f96700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35000b4ee0 con 0x7f350015a190 2023-07-08T12:38:59.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f3504f96700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35000cedd0 con 0x7f35000acb30 2023-07-08T12:38:59.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f34fdd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35000b12b0 0x7f35000b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f34fdd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35000b12b0 0x7f35000b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50602/0 (socket says 172.21.15.12:50602) 2023-07-08T12:38:59.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f34fdd9b700 1 -- 172.21.15.12:0/1177482494 learned_addr learned my addr 172.21.15.12:0/1177482494 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:59.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f34fe59c700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35000acb30 0x7f3500154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f34fed9d700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f350015a190 0x7f35000b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f34fdd9b700 1 -- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35000acb30 msgr2=0x7f3500154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f34fdd9b700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35000acb30 0x7f3500154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f34fdd9b700 1 -- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f350015a190 msgr2=0x7f35000b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f34fdd9b700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f350015a190 0x7f35000b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f34fdd9b700 1 -- 172.21.15.12:0/1177482494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f8009b30 con 0x7f35000b12b0 2023-07-08T12:38:59.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.485+0000 7f34fed9d700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f350015a190 0x7f35000b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:38:59.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.486+0000 7f34fdd9b700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35000b12b0 0x7f35000b4460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c550 tx=0x7f34f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:59.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.486+0000 7f34eeffd700 1 -- 172.21.15.12:0/1177482494 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0004c80 con 0x7f35000b12b0 2023-07-08T12:38:59.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.486+0000 7f34eeffd700 1 -- 172.21.15.12:0/1177482494 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f0004e00 con 0x7f35000b12b0 2023-07-08T12:38:59.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.486+0000 7f34eeffd700 1 -- 172.21.15.12:0/1177482494 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0007310 con 0x7f35000b12b0 2023-07-08T12:38:59.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.486+0000 7f3504f96700 1 -- 172.21.15.12:0/1177482494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35000c3870 con 0x7f35000b12b0 2023-07-08T12:38:59.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.486+0000 7f3504f96700 1 -- 172.21.15.12:0/1177482494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3500155330 con 0x7f35000b12b0 2023-07-08T12:38:59.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.487+0000 7f3504f96700 1 -- 172.21.15.12:0/1177482494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35000c4070 con 0x7f35000b12b0 2023-07-08T12:38:59.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.487+0000 7f34eeffd700 1 -- 172.21.15.12:0/1177482494 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f34f0007470 con 0x7f35000b12b0 2023-07-08T12:38:59.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.488+0000 7f34eeffd700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f34e006ace0 0x7f34e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.488+0000 7f34eeffd700 1 -- 172.21.15.12:0/1177482494 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f34f0089480 con 0x7f35000b12b0 2023-07-08T12:38:59.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.488+0000 7f34fe59c700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f34e006ace0 0x7f34e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.488+0000 7f34fe59c700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f34e006ace0 0x7f34e006d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f35001f53b0 tx=0x7f34f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:59.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.492+0000 7f34eeffd700 1 -- 172.21.15.12:0/1177482494 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f35000c4070 con 0x7f35000b12b0 2023-07-08T12:38:59.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.687+0000 7f3504f96700 1 -- 172.21.15.12:0/1177482494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f35000c4070 con 0x7f35000b12b0 2023-07-08T12:38:59.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.687+0000 7f34eeffd700 1 -- 172.21.15.12:0/1177482494 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f35000c4070 con 0x7f35000b12b0 2023-07-08T12:38:59.690 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:38:59.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.689+0000 7f3504f96700 1 -- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f34e006ace0 msgr2=0x7f34e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.689+0000 7f3504f96700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f34e006ace0 0x7f34e006d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f35001f53b0 tx=0x7f34f4008040 comp rx=0 tx=0).stop 2023-07-08T12:38:59.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.689+0000 7f3504f96700 1 -- 172.21.15.12:0/1177482494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35000b12b0 msgr2=0x7f35000b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.689+0000 7f3504f96700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35000b12b0 0x7f35000b4460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c550 tx=0x7f34f000c910 comp rx=0 tx=0).stop 2023-07-08T12:38:59.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.689+0000 7f3504f96700 1 -- 172.21.15.12:0/1177482494 shutdown_connections 2023-07-08T12:38:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.689+0000 7f3504f96700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f34e006ace0 0x7f34e006d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.689+0000 7f3504f96700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f350015a190 0x7f35000b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.689+0000 7f3504f96700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35000acb30 0x7f3500154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.689+0000 7f3504f96700 1 --2- 172.21.15.12:0/1177482494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35000b12b0 0x7f35000b4460 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.689+0000 7f3504f96700 1 -- 172.21.15.12:0/1177482494 >> 172.21.15.12:0/1177482494 conn(0x7f3500098b50 msgr2=0x7f35000af290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.689+0000 7f3504f96700 1 -- 172.21.15.12:0/1177482494 shutdown_connections 2023-07-08T12:38:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.689+0000 7f3504f96700 1 -- 172.21.15.12:0/1177482494 wait complete. 2023-07-08T12:38:59.704 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-08T12:38:59.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.830+0000 7f4a83152700 1 Processor -- start 2023-07-08T12:38:59.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.830+0000 7f4a83152700 1 -- start start 2023-07-08T12:38:59.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a83152700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a7c0aeec0 0x7f4a7c06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a83152700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a7c06ebe0 0x7f4a7c0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a83152700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4a7c0cd9b0 0x7f4a7c0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a83152700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a7c15d770 con 0x7f4a7c06ebe0 2023-07-08T12:38:59.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a83152700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a7c0cddf0 con 0x7f4a7c0cd9b0 2023-07-08T12:38:59.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a83152700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c15cb60 con 0x7f4a7c0aeec0 2023-07-08T12:38:59.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a7bfff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a7c06ebe0 0x7f4a7c0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a7bfff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a7c06ebe0 0x7f4a7c0cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50608/0 (socket says 172.21.15.12:50608) 2023-07-08T12:38:59.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a7bfff700 1 -- 172.21.15.12:0/372107523 learned_addr learned my addr 172.21.15.12:0/372107523 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:59.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a80eee700 1 --2- 172.21.15.12:0/372107523 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a7c0aeec0 0x7f4a7c06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a816ef700 1 --2- 172.21.15.12:0/372107523 >> v2:172.21.15.160:3300/0 conn(0x7f4a7c0cd9b0 0x7f4a7c0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a7bfff700 1 -- 172.21.15.12:0/372107523 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a7c0aeec0 msgr2=0x7f4a7c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:59.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a7bfff700 1 --2- 172.21.15.12:0/372107523 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a7c0aeec0 0x7f4a7c06cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a7bfff700 1 -- 172.21.15.12:0/372107523 >> v2:172.21.15.160:3300/0 conn(0x7f4a7c0cd9b0 msgr2=0x7f4a7c0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:38:59.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a7bfff700 1 --2- 172.21.15.12:0/372107523 >> v2:172.21.15.160:3300/0 conn(0x7f4a7c0cd9b0 0x7f4a7c0cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a7bfff700 1 -- 172.21.15.12:0/372107523 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7c15ce40 con 0x7f4a7c06ebe0 2023-07-08T12:38:59.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a816ef700 1 --2- 172.21.15.12:0/372107523 >> v2:172.21.15.160:3300/0 conn(0x7f4a7c0cd9b0 0x7f4a7c0cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:38:59.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a7bfff700 1 --2- 172.21.15.12:0/372107523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a7c06ebe0 0x7f4a7c0cd470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0ce0c0 tx=0x7f4a68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42958753c37a2388 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:59.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a7b7fe700 1 -- 172.21.15.12:0/372107523 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68011830 con 0x7f4a7c06ebe0 2023-07-08T12:38:59.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.832+0000 7f4a7b7fe700 1 -- 172.21.15.12:0/372107523 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a68011990 con 0x7f4a7c06ebe0 2023-07-08T12:38:59.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.833+0000 7f4a7b7fe700 1 -- 172.21.15.12:0/372107523 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68011d20 con 0x7f4a7c06ebe0 2023-07-08T12:38:59.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.833+0000 7f4a83152700 1 -- 172.21.15.12:0/372107523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a7c06ebe0 msgr2=0x7f4a7c0cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.833+0000 7f4a83152700 1 --2- 172.21.15.12:0/372107523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a7c06ebe0 0x7f4a7c0cd470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0ce0c0 tx=0x7f4a68009bd0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.833+0000 7f4a83152700 1 -- 172.21.15.12:0/372107523 shutdown_connections 2023-07-08T12:38:59.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.833+0000 7f4a83152700 1 --2- 172.21.15.12:0/372107523 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a7c0aeec0 0x7f4a7c06cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.833+0000 7f4a83152700 1 --2- 172.21.15.12:0/372107523 >> v2:172.21.15.160:3300/0 conn(0x7f4a7c0cd9b0 0x7f4a7c0cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.833+0000 7f4a83152700 1 --2- 172.21.15.12:0/372107523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a7c06ebe0 0x7f4a7c0cd470 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.833+0000 7f4a83152700 1 -- 172.21.15.12:0/372107523 >> 172.21.15.12:0/372107523 conn(0x7f4a7c098b50 msgr2=0x7f4a7c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:38:59.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.833+0000 7f4a83152700 1 -- 172.21.15.12:0/372107523 shutdown_connections 2023-07-08T12:38:59.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.833+0000 7f4a83152700 1 -- 172.21.15.12:0/372107523 wait complete. 2023-07-08T12:38:59.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.834+0000 7f4a83152700 1 Processor -- start 2023-07-08T12:38:59.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.834+0000 7f4a83152700 1 -- start start 2023-07-08T12:38:59.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.835+0000 7f4a83152700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a7c06ebe0 0x7f4a7c147800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.835+0000 7f4a83152700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a7c0aeec0 0x7f4a7c0cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.835+0000 7f4a83152700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a7c0cd9b0 0x7f4a7c152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.835+0000 7f4a83152700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c0cc2e0 con 0x7f4a7c0aeec0 2023-07-08T12:38:59.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.835+0000 7f4a83152700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c14f350 con 0x7f4a7c0cd9b0 2023-07-08T12:38:59.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.835+0000 7f4a83152700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c0cb660 con 0x7f4a7c06ebe0 2023-07-08T12:38:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.835+0000 7f4a7bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a7c0aeec0 0x7f4a7c0cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.835+0000 7f4a7bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a7c0aeec0 0x7f4a7c0cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50614/0 (socket says 172.21.15.12:50614) 2023-07-08T12:38:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.835+0000 7f4a7bfff700 1 -- 172.21.15.12:0/3212532878 learned_addr learned my addr 172.21.15.12:0/3212532878 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:38:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.836+0000 7f4a80eee700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a7c06ebe0 0x7f4a7c147800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.836+0000 7f4a816ef700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a7c0cd9b0 0x7f4a7c152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.836+0000 7f4a7bfff700 1 -- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a7c06ebe0 msgr2=0x7f4a7c147800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.836+0000 7f4a7bfff700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a7c06ebe0 0x7f4a7c147800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.836+0000 7f4a7bfff700 1 -- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a7c0cd9b0 msgr2=0x7f4a7c152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:38:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.836+0000 7f4a7bfff700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a7c0cd9b0 0x7f4a7c152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:38:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.836+0000 7f4a7bfff700 1 -- 172.21.15.12:0/3212532878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a68009550 con 0x7f4a7c0aeec0 2023-07-08T12:38:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.836+0000 7f4a7bfff700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a7c0aeec0 0x7f4a7c0cac80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4a68000c00 tx=0x7f4a68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.836+0000 7f4a797fa700 1 -- 172.21.15.12:0/3212532878 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68004380 con 0x7f4a7c0aeec0 2023-07-08T12:38:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.836+0000 7f4a797fa700 1 -- 172.21.15.12:0/3212532878 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6801f070 con 0x7f4a7c0aeec0 2023-07-08T12:38:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.836+0000 7f4a83152700 1 -- 172.21.15.12:0/3212532878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7c0cb940 con 0x7f4a7c0aeec0 2023-07-08T12:38:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.836+0000 7f4a797fa700 1 -- 172.21.15.12:0/3212532878 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a680036a0 con 0x7f4a7c0aeec0 2023-07-08T12:38:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.836+0000 7f4a83152700 1 -- 172.21.15.12:0/3212532878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7c0cbf00 con 0x7f4a7c0aeec0 2023-07-08T12:38:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.837+0000 7f4a83152700 1 -- 172.21.15.12:0/3212532878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a7c0cc420 con 0x7f4a7c0aeec0 2023-07-08T12:38:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.837+0000 7f4a797fa700 1 -- 172.21.15.12:0/3212532878 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a68021030 con 0x7f4a7c0aeec0 2023-07-08T12:38:59.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.838+0000 7f4a797fa700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a6006ace0 0x7f4a6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:38:59.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.838+0000 7f4a797fa700 1 -- 172.21.15.12:0/3212532878 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4a6808acf0 con 0x7f4a7c0aeec0 2023-07-08T12:38:59.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.838+0000 7f4a80eee700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a6006ace0 0x7f4a6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:38:59.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.839+0000 7f4a80eee700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a6006ace0 0x7f4a6006d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c1670e0 tx=0x7f4a70008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:38:59.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:38:59.842+0000 7f4a797fa700 1 -- 172.21.15.12:0/3212532878 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4a7c0cc420 con 0x7f4a7c0aeec0 2023-07-08T12:39:00.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.042+0000 7f4a83152700 1 -- 172.21.15.12:0/3212532878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a7c0cc420 con 0x7f4a7c0aeec0 2023-07-08T12:39:00.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.042+0000 7f4a797fa700 1 -- 172.21.15.12:0/3212532878 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a7c0cc420 con 0x7f4a7c0aeec0 2023-07-08T12:39:00.045 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:00.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.044+0000 7f4a83152700 1 -- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a6006ace0 msgr2=0x7f4a6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.044+0000 7f4a83152700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a6006ace0 0x7f4a6006d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c1670e0 tx=0x7f4a70008040 comp rx=0 tx=0).stop 2023-07-08T12:39:00.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.044+0000 7f4a83152700 1 -- 172.21.15.12:0/3212532878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a7c0aeec0 msgr2=0x7f4a7c0cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.044+0000 7f4a83152700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a7c0aeec0 0x7f4a7c0cac80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4a68000c00 tx=0x7f4a68004750 comp rx=0 tx=0).stop 2023-07-08T12:39:00.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.044+0000 7f4a83152700 1 -- 172.21.15.12:0/3212532878 shutdown_connections 2023-07-08T12:39:00.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.044+0000 7f4a83152700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a6006ace0 0x7f4a6006d190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.044+0000 7f4a83152700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a7c0cd9b0 0x7f4a7c152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.044+0000 7f4a83152700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a7c06ebe0 0x7f4a7c147800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.044+0000 7f4a83152700 1 --2- 172.21.15.12:0/3212532878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a7c0aeec0 0x7f4a7c0cac80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.044+0000 7f4a83152700 1 -- 172.21.15.12:0/3212532878 >> 172.21.15.12:0/3212532878 conn(0x7f4a7c098b50 msgr2=0x7f4a7c15b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:00.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.045+0000 7f4a83152700 1 -- 172.21.15.12:0/3212532878 shutdown_connections 2023-07-08T12:39:00.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.045+0000 7f4a83152700 1 -- 172.21.15.12:0/3212532878 wait complete. 2023-07-08T12:39:00.059 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-08T12:39:00.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.184+0000 7fa4969f5700 1 Processor -- start 2023-07-08T12:39:00.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.185+0000 7fa4969f5700 1 -- start start 2023-07-08T12:39:00.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.185+0000 7fa4969f5700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa4900aeec0 0x7fa49006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.185+0000 7fa4969f5700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4900cdb80 0x7fa4900cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.185+0000 7fa4969f5700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49015d770 con 0x7fa49006ebe0 2023-07-08T12:39:00.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.185+0000 7fa4969f5700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49015a600 con 0x7fa4900aeec0 2023-07-08T12:39:00.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.185+0000 7fa4969f5700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49015cb60 con 0x7fa4900cdb80 2023-07-08T12:39:00.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.185+0000 7fa48ffff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa4900aeec0 0x7fa49006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.185+0000 7fa48f7fe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa49006ebe0 0x7fa4900cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38016/0 (socket says 172.21.15.12:38016) 2023-07-08T12:39:00.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.185+0000 7fa48f7fe700 1 -- 172.21.15.12:0/2574765480 learned_addr learned my addr 172.21.15.12:0/2574765480 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.186+0000 7fa494f92700 1 --2- 172.21.15.12:0/2574765480 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4900cdb80 0x7fa4900cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.186+0000 7fa48ffff700 1 -- 172.21.15.12:0/2574765480 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4900cdb80 msgr2=0x7fa4900cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:00.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.186+0000 7fa48ffff700 1 --2- 172.21.15.12:0/2574765480 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4900cdb80 0x7fa4900cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.186+0000 7fa48ffff700 1 -- 172.21.15.12:0/2574765480 >> v1:172.21.15.12:6789/0 conn(0x7fa49006ebe0 legacy=0x7fa4900cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.186+0000 7fa48ffff700 1 -- 172.21.15.12:0/2574765480 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49015ce40 con 0x7fa4900aeec0 2023-07-08T12:39:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.186+0000 7fa48effd700 1 -- 172.21.15.12:0/2574765480 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378907992 0 0) 0x7fa49015d770 con 0x7fa49006ebe0 2023-07-08T12:39:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.186+0000 7fa494f92700 1 --2- 172.21.15.12:0/2574765480 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4900cdb80 0x7fa4900cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.186+0000 7fa48ffff700 1 --2- 172.21.15.12:0/2574765480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900aeec0 0x7fa49006cff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa47800bac0 tx=0x7fa47800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcaf0e129ef858c server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.186+0000 7fa48effd700 1 -- 172.21.15.12:0/2574765480 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa478005e30 con 0x7fa4900aeec0 2023-07-08T12:39:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.186+0000 7fa48effd700 1 -- 172.21.15.12:0/2574765480 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47800d920 con 0x7fa4900aeec0 2023-07-08T12:39:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.186+0000 7fa48effd700 1 -- 172.21.15.12:0/2574765480 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47800dcb0 con 0x7fa4900aeec0 2023-07-08T12:39:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.187+0000 7fa4969f5700 1 -- 172.21.15.12:0/2574765480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900aeec0 msgr2=0x7fa49006cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.187+0000 7fa4969f5700 1 --2- 172.21.15.12:0/2574765480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900aeec0 0x7fa49006cff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa47800bac0 tx=0x7fa47800bdd0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.187+0000 7fa4969f5700 1 -- 172.21.15.12:0/2574765480 shutdown_connections 2023-07-08T12:39:00.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.187+0000 7fa4969f5700 1 --2- 172.21.15.12:0/2574765480 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4900cdb80 0x7fa4900cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.187+0000 7fa4969f5700 1 --2- 172.21.15.12:0/2574765480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900aeec0 0x7fa49006cff0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.187+0000 7fa4969f5700 1 -- 172.21.15.12:0/2574765480 >> 172.21.15.12:0/2574765480 conn(0x7fa490098b50 msgr2=0x7fa49006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:00.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.187+0000 7fa4969f5700 1 -- 172.21.15.12:0/2574765480 shutdown_connections 2023-07-08T12:39:00.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.187+0000 7fa4969f5700 1 -- 172.21.15.12:0/2574765480 wait complete. 2023-07-08T12:39:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.188+0000 7fa4969f5700 1 Processor -- start 2023-07-08T12:39:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.188+0000 7fa4969f5700 1 -- start start 2023-07-08T12:39:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa4969f5700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa49006ebe0 0x7fa4900cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa4969f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900aeec0 0x7fa4900c2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa4969f5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4900cdb80 0x7fa490156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa4969f5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900ca550 con 0x7fa4900aeec0 2023-07-08T12:39:00.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa4969f5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900ca830 con 0x7fa4900cdb80 2023-07-08T12:39:00.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa4969f5700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900cab10 con 0x7fa49006ebe0 2023-07-08T12:39:00.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa48f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900aeec0 0x7fa4900c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa48f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900aeec0 0x7fa4900c2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50628/0 (socket says 172.21.15.12:50628) 2023-07-08T12:39:00.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa48f7fe700 1 -- 172.21.15.12:0/1675876570 learned_addr learned my addr 172.21.15.12:0/1675876570 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:00.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa494f92700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4900cdb80 0x7fa490156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa48ffff700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa49006ebe0 0x7fa4900cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa48f7fe700 1 -- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa49006ebe0 msgr2=0x7fa4900cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa48f7fe700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa49006ebe0 0x7fa4900cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa48f7fe700 1 -- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4900cdb80 msgr2=0x7fa490156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa48f7fe700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4900cdb80 0x7fa490156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.189+0000 7fa48f7fe700 1 -- 172.21.15.12:0/1675876570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47800b6d0 con 0x7fa4900aeec0 2023-07-08T12:39:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.190+0000 7fa48f7fe700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900aeec0 0x7fa4900c2740 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa4800011e0 tx=0x7fa48000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.190+0000 7fa48cff9700 1 -- 172.21.15.12:0/1675876570 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480004c80 con 0x7fa4900aeec0 2023-07-08T12:39:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.190+0000 7fa48cff9700 1 -- 172.21.15.12:0/1675876570 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa480004e00 con 0x7fa4900aeec0 2023-07-08T12:39:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.190+0000 7fa4969f5700 1 -- 172.21.15.12:0/1675876570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4900cadf0 con 0x7fa4900aeec0 2023-07-08T12:39:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.190+0000 7fa48cff9700 1 -- 172.21.15.12:0/1675876570 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480007310 con 0x7fa4900aeec0 2023-07-08T12:39:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.190+0000 7fa4969f5700 1 -- 172.21.15.12:0/1675876570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4900c9fe0 con 0x7fa4900aeec0 2023-07-08T12:39:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.191+0000 7fa4969f5700 1 -- 172.21.15.12:0/1675876570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4900cf0f0 con 0x7fa4900aeec0 2023-07-08T12:39:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.191+0000 7fa48cff9700 1 -- 172.21.15.12:0/1675876570 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa480007470 con 0x7fa4900aeec0 2023-07-08T12:39:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.192+0000 7fa48cff9700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa47006ace0 0x7fa47006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.192+0000 7fa48cff9700 1 -- 172.21.15.12:0/1675876570 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa480089530 con 0x7fa4900aeec0 2023-07-08T12:39:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.192+0000 7fa48ffff700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa47006ace0 0x7fa47006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.192+0000 7fa48ffff700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa47006ace0 0x7fa47006d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa47800be10 tx=0x7fa478008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.196+0000 7fa48cff9700 1 -- 172.21.15.12:0/1675876570 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa4800542b0 con 0x7fa4900aeec0 2023-07-08T12:39:00.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.392+0000 7fa4969f5700 1 -- 172.21.15.12:0/1675876570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa49004ea60 con 0x7fa4900aeec0 2023-07-08T12:39:00.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.393+0000 7fa48cff9700 1 -- 172.21.15.12:0/1675876570 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4800578d0 con 0x7fa4900aeec0 2023-07-08T12:39:00.399 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-07-08T12:39:00.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.395+0000 7fa4969f5700 1 -- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa47006ace0 msgr2=0x7fa47006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.395+0000 7fa4969f5700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa47006ace0 0x7fa47006d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa47800be10 tx=0x7fa478008040 comp rx=0 tx=0).stop 2023-07-08T12:39:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.395+0000 7fa4969f5700 1 -- 172.21.15.12:0/1675876570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900aeec0 msgr2=0x7fa4900c2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.395+0000 7fa4969f5700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900aeec0 0x7fa4900c2740 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa4800011e0 tx=0x7fa48000cae0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.395+0000 7fa4969f5700 1 -- 172.21.15.12:0/1675876570 shutdown_connections 2023-07-08T12:39:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.395+0000 7fa4969f5700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa47006ace0 0x7fa47006d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.395+0000 7fa4969f5700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4900cdb80 0x7fa490156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.395+0000 7fa4969f5700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa49006ebe0 0x7fa4900cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.395+0000 7fa4969f5700 1 --2- 172.21.15.12:0/1675876570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4900aeec0 0x7fa4900c2740 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.395+0000 7fa4969f5700 1 -- 172.21.15.12:0/1675876570 >> 172.21.15.12:0/1675876570 conn(0x7fa490098b50 msgr2=0x7fa4900cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:00.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.395+0000 7fa4969f5700 1 -- 172.21.15.12:0/1675876570 shutdown_connections 2023-07-08T12:39:00.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.395+0000 7fa4969f5700 1 -- 172.21.15.12:0/1675876570 wait complete. 2023-07-08T12:39:00.409 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-08T12:39:00.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.535+0000 7fbedbda6700 1 Processor -- start 2023-07-08T12:39:00.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.535+0000 7fbedbda6700 1 -- start start 2023-07-08T12:39:00.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.536+0000 7fbedbda6700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbed4156a10 0x7fbed406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.536+0000 7fbedbda6700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbed40cc240 0x7fbed40cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.536+0000 7fbedbda6700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed4160410 con 0x7fbed415d390 2023-07-08T12:39:00.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.536+0000 7fbedbda6700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed40ccc00 con 0x7fbed40cc240 2023-07-08T12:39:00.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.536+0000 7fbedbda6700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed4156de0 con 0x7fbed4156a10 2023-07-08T12:39:00.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.536+0000 7fbeda343700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbed40cc240 0x7fbed40cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.536+0000 7fbed9341700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbed415d390 0x7fbed41608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38030/0 (socket says 172.21.15.12:38030) 2023-07-08T12:39:00.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.536+0000 7fbed9341700 1 -- 172.21.15.12:0/1968422494 learned_addr learned my addr 172.21.15.12:0/1968422494 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:00.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.536+0000 7fbed9b42700 1 --2- 172.21.15.12:0/1968422494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbed4156a10 0x7fbed406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbeda343700 1 -- 172.21.15.12:0/1968422494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbed4156a10 msgr2=0x7fbed406e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:00.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbeda343700 1 --2- 172.21.15.12:0/1968422494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbed4156a10 0x7fbed406e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbeda343700 1 -- 172.21.15.12:0/1968422494 >> v1:172.21.15.12:6789/0 conn(0x7fbed415d390 legacy=0x7fbed41608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbed8b40700 1 -- 172.21.15.12:0/1968422494 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171644168 0 0) 0x7fbed4160410 con 0x7fbed415d390 2023-07-08T12:39:00.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbeda343700 1 -- 172.21.15.12:0/1968422494 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed406ec70 con 0x7fbed40cc240 2023-07-08T12:39:00.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbed9b42700 1 --2- 172.21.15.12:0/1968422494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbed4156a10 0x7fbed406e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:00.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbeda343700 1 --2- 172.21.15.12:0/1968422494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbed40cc240 0x7fbed40cc6c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009d90 tx=0x7fbecc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1826062d70276c13 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:00.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbed8b40700 1 -- 172.21.15.12:0/1968422494 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc00e070 con 0x7fbed40cc240 2023-07-08T12:39:00.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbed8b40700 1 -- 172.21.15.12:0/1968422494 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc00bbd0 con 0x7fbed40cc240 2023-07-08T12:39:00.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbed8b40700 1 -- 172.21.15.12:0/1968422494 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc012430 con 0x7fbed40cc240 2023-07-08T12:39:00.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbedbda6700 1 -- 172.21.15.12:0/1968422494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbed40cc240 msgr2=0x7fbed40cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbedbda6700 1 --2- 172.21.15.12:0/1968422494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbed40cc240 0x7fbed40cc6c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009d90 tx=0x7fbecc00b330 comp rx=0 tx=0).stop 2023-07-08T12:39:00.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbedbda6700 1 -- 172.21.15.12:0/1968422494 shutdown_connections 2023-07-08T12:39:00.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbedbda6700 1 --2- 172.21.15.12:0/1968422494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbed40cc240 0x7fbed40cc6c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbedbda6700 1 --2- 172.21.15.12:0/1968422494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbed4156a10 0x7fbed406e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.537+0000 7fbedbda6700 1 -- 172.21.15.12:0/1968422494 >> 172.21.15.12:0/1968422494 conn(0x7fbed4098b50 msgr2=0x7fbed406b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:00.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.538+0000 7fbedbda6700 1 -- 172.21.15.12:0/1968422494 shutdown_connections 2023-07-08T12:39:00.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.538+0000 7fbedbda6700 1 -- 172.21.15.12:0/1968422494 wait complete. 2023-07-08T12:39:00.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.539+0000 7fbedbda6700 1 Processor -- start 2023-07-08T12:39:00.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.539+0000 7fbedbda6700 1 -- start start 2023-07-08T12:39:00.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.539+0000 7fbedbda6700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbed40cc240 0x7fbed4146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.539+0000 7fbedbda6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbed4156a10 0x7fbed41f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.539+0000 7fbedbda6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbed415d390 0x7fbed4156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.539+0000 7fbedbda6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed4149410 con 0x7fbed4156a10 2023-07-08T12:39:00.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.539+0000 7fbedbda6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed4149130 con 0x7fbed415d390 2023-07-08T12:39:00.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.539+0000 7fbedbda6700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed415d930 con 0x7fbed40cc240 2023-07-08T12:39:00.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbed9341700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbed4156a10 0x7fbed41f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbed9341700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbed4156a10 0x7fbed41f2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50642/0 (socket says 172.21.15.12:50642) 2023-07-08T12:39:00.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbed9341700 1 -- 172.21.15.12:0/560898954 learned_addr learned my addr 172.21.15.12:0/560898954 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:00.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbed9b42700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbed40cc240 0x7fbed4146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbeda343700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbed415d390 0x7fbed4156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbed9341700 1 -- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbed40cc240 msgr2=0x7fbed4146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbed9341700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbed40cc240 0x7fbed4146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbed9341700 1 -- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbed415d390 msgr2=0x7fbed4156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbed9341700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbed415d390 0x7fbed4156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbed9341700 1 -- 172.21.15.12:0/560898954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc009a70 con 0x7fbed4156a10 2023-07-08T12:39:00.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbed9341700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbed4156a10 0x7fbed41f2450 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbec40011e0 tx=0x7fbec400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:00.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbeca7fc700 1 -- 172.21.15.12:0/560898954 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4004c80 con 0x7fbed4156a10 2023-07-08T12:39:00.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbeca7fc700 1 -- 172.21.15.12:0/560898954 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec4004e00 con 0x7fbed4156a10 2023-07-08T12:39:00.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbedbda6700 1 -- 172.21.15.12:0/560898954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed415dc10 con 0x7fbed4156a10 2023-07-08T12:39:00.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbeca7fc700 1 -- 172.21.15.12:0/560898954 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4007310 con 0x7fbed4156a10 2023-07-08T12:39:00.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.540+0000 7fbedbda6700 1 -- 172.21.15.12:0/560898954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed41f2990 con 0x7fbed4156a10 2023-07-08T12:39:00.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.541+0000 7fbedbda6700 1 -- 172.21.15.12:0/560898954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed415e130 con 0x7fbed4156a10 2023-07-08T12:39:00.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.542+0000 7fbeca7fc700 1 -- 172.21.15.12:0/560898954 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbec4007470 con 0x7fbed4156a10 2023-07-08T12:39:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.542+0000 7fbeca7fc700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbeb806ace0 0x7fbeb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.542+0000 7fbeca7fc700 1 -- 172.21.15.12:0/560898954 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbec4089550 con 0x7fbed4156a10 2023-07-08T12:39:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.542+0000 7fbed9b42700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbeb806ace0 0x7fbeb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.543+0000 7fbed9b42700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbeb806ace0 0x7fbeb806d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbed000aa50 tx=0x7fbed0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:00.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.546+0000 7fbeca7fc700 1 -- 172.21.15.12:0/560898954 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbed415e130 con 0x7fbed4156a10 2023-07-08T12:39:00.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.747+0000 7fbedbda6700 1 -- 172.21.15.12:0/560898954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbed415e130 con 0x7fbed4156a10 2023-07-08T12:39:00.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.748+0000 7fbeca7fc700 1 -- 172.21.15.12:0/560898954 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbed415e130 con 0x7fbed4156a10 2023-07-08T12:39:00.750 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:00.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.750+0000 7fbedbda6700 1 -- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbeb806ace0 msgr2=0x7fbeb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.750+0000 7fbedbda6700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbeb806ace0 0x7fbeb806d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbed000aa50 tx=0x7fbed0008040 comp rx=0 tx=0).stop 2023-07-08T12:39:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.750+0000 7fbedbda6700 1 -- 172.21.15.12:0/560898954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbed4156a10 msgr2=0x7fbed41f2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.750+0000 7fbedbda6700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbed4156a10 0x7fbed41f2450 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbec40011e0 tx=0x7fbec400cae0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.750+0000 7fbedbda6700 1 -- 172.21.15.12:0/560898954 shutdown_connections 2023-07-08T12:39:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.750+0000 7fbedbda6700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbeb806ace0 0x7fbeb806d190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.750+0000 7fbedbda6700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbed415d390 0x7fbed4156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.750+0000 7fbedbda6700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbed40cc240 0x7fbed4146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.750+0000 7fbedbda6700 1 --2- 172.21.15.12:0/560898954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbed4156a10 0x7fbed41f2450 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.750+0000 7fbedbda6700 1 -- 172.21.15.12:0/560898954 >> 172.21.15.12:0/560898954 conn(0x7fbed4098b50 msgr2=0x7fbed40ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:00.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.750+0000 7fbedbda6700 1 -- 172.21.15.12:0/560898954 shutdown_connections 2023-07-08T12:39:00.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.750+0000 7fbedbda6700 1 -- 172.21.15.12:0/560898954 wait complete. 2023-07-08T12:39:00.765 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:39:00.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.887+0000 7fb0bad7b700 1 Processor -- start 2023-07-08T12:39:00.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.887+0000 7fb0bad7b700 1 -- start start 2023-07-08T12:39:00.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.888+0000 7fb0bad7b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb0b415a480 0x7fb0b40cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.888+0000 7fb0bad7b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b40cfea0 0x7fb0b40b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.888+0000 7fb0bad7b700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b415a850 con 0x7fb0b40b2410 2023-07-08T12:39:00.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.888+0000 7fb0bad7b700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b40b2da0 con 0x7fb0b415a480 2023-07-08T12:39:00.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.888+0000 7fb0bad7b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b40cde90 con 0x7fb0b40cfea0 2023-07-08T12:39:00.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.888+0000 7fb0b8b17700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb0b415a480 0x7fb0b40cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.888+0000 7fb0b8b17700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb0b415a480 0x7fb0b40cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50648/0 (socket says 172.21.15.12:50648) 2023-07-08T12:39:00.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.888+0000 7fb0b8b17700 1 -- 172.21.15.12:0/4293889089 learned_addr learned my addr 172.21.15.12:0/4293889089 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:00.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0b9318700 1 --2- 172.21.15.12:0/4293889089 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b40cfea0 0x7fb0b40b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0b8b17700 1 -- 172.21.15.12:0/4293889089 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b40cfea0 msgr2=0x7fb0b40b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:00.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0b8b17700 1 --2- 172.21.15.12:0/4293889089 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b40cfea0 0x7fb0b40b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0b8b17700 1 -- 172.21.15.12:0/4293889089 >> v1:172.21.15.160:6789/0 conn(0x7fb0b40b2410 legacy=0x7fb0b415def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:00.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0b8b17700 1 -- 172.21.15.12:0/4293889089 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b40cd8d0 con 0x7fb0b415a480 2023-07-08T12:39:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0b9318700 1 --2- 172.21.15.12:0/4293889089 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b40cfea0 0x7fb0b40b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0b8b17700 1 --2- 172.21.15.12:0/4293889089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0b415a480 0x7fb0b40cafe0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80098f0 tx=0x7fb0a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c631b167b8289a34 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0b37fe700 1 -- 172.21.15.12:0/4293889089 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8004230 con 0x7fb0b415a480 2023-07-08T12:39:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0b37fe700 1 -- 172.21.15.12:0/4293889089 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a8004390 con 0x7fb0b415a480 2023-07-08T12:39:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0b37fe700 1 -- 172.21.15.12:0/4293889089 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a80036a0 con 0x7fb0b415a480 2023-07-08T12:39:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0bad7b700 1 -- 172.21.15.12:0/4293889089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0b415a480 msgr2=0x7fb0b40cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0bad7b700 1 --2- 172.21.15.12:0/4293889089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0b415a480 0x7fb0b40cafe0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80098f0 tx=0x7fb0a8009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:00.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0bad7b700 1 -- 172.21.15.12:0/4293889089 shutdown_connections 2023-07-08T12:39:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0bad7b700 1 --2- 172.21.15.12:0/4293889089 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b40cfea0 0x7fb0b40b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0bad7b700 1 --2- 172.21.15.12:0/4293889089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0b415a480 0x7fb0b40cafe0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.889+0000 7fb0bad7b700 1 -- 172.21.15.12:0/4293889089 >> 172.21.15.12:0/4293889089 conn(0x7fb0b406ab60 msgr2=0x7fb0b406e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.890+0000 7fb0bad7b700 1 -- 172.21.15.12:0/4293889089 shutdown_connections 2023-07-08T12:39:00.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.890+0000 7fb0bad7b700 1 -- 172.21.15.12:0/4293889089 wait complete. 2023-07-08T12:39:00.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.891+0000 7fb0bad7b700 1 Processor -- start 2023-07-08T12:39:00.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.891+0000 7fb0bad7b700 1 -- start start 2023-07-08T12:39:00.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.892+0000 7fb0bad7b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b40b2410 0x7fb0b4148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.892+0000 7fb0bad7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0b40cfea0 0x7fb0b41e1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.892+0000 7fb0bad7b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0b415a480 0x7fb0b41df700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.892+0000 7fb0bad7b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b41e0e80 con 0x7fb0b40cfea0 2023-07-08T12:39:00.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.892+0000 7fb0bad7b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b4159ad0 con 0x7fb0b415a480 2023-07-08T12:39:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.892+0000 7fb0bad7b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b41e0270 con 0x7fb0b40b2410 2023-07-08T12:39:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.892+0000 7fb0b3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0b40cfea0 0x7fb0b41e1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.892+0000 7fb0b3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0b40cfea0 0x7fb0b41e1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50656/0 (socket says 172.21.15.12:50656) 2023-07-08T12:39:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.892+0000 7fb0b3fff700 1 -- 172.21.15.12:0/2640337313 learned_addr learned my addr 172.21.15.12:0/2640337313 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0b8b17700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b40b2410 0x7fb0b4148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0b9318700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0b415a480 0x7fb0b41df700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0b3fff700 1 -- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b40b2410 msgr2=0x7fb0b4148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0b3fff700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b40b2410 0x7fb0b4148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0b3fff700 1 -- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0b415a480 msgr2=0x7fb0b41df700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0b3fff700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0b415a480 0x7fb0b41df700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0b3fff700 1 -- 172.21.15.12:0/2640337313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a8009550 con 0x7fb0b40cfea0 2023-07-08T12:39:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0b9318700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0b415a480 0x7fb0b41df700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0b3fff700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0b40cfea0 0x7fb0b41e1310 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000b4a0 tx=0x7fb0a000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0b17fa700 1 -- 172.21.15.12:0/2640337313 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0011830 con 0x7fb0b40cfea0 2023-07-08T12:39:00.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0b17fa700 1 -- 172.21.15.12:0/2640337313 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a0011990 con 0x7fb0b40cfea0 2023-07-08T12:39:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0bad7b700 1 -- 172.21.15.12:0/2640337313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b41e0550 con 0x7fb0b40cfea0 2023-07-08T12:39:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0b17fa700 1 -- 172.21.15.12:0/2640337313 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0011d40 con 0x7fb0b40cfea0 2023-07-08T12:39:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.893+0000 7fb0bad7b700 1 -- 172.21.15.12:0/2640337313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b41e0b10 con 0x7fb0b40cfea0 2023-07-08T12:39:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.894+0000 7fb0bad7b700 1 -- 172.21.15.12:0/2640337313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b41e0ff0 con 0x7fb0b40cfea0 2023-07-08T12:39:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.895+0000 7fb0b17fa700 1 -- 172.21.15.12:0/2640337313 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb0a001a400 con 0x7fb0b40cfea0 2023-07-08T12:39:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.895+0000 7fb0b17fa700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb09806ace0 0x7fb09806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:00.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.895+0000 7fb0b17fa700 1 -- 172.21.15.12:0/2640337313 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb0a0089460 con 0x7fb0b40cfea0 2023-07-08T12:39:00.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.895+0000 7fb0b8b17700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb09806ace0 0x7fb09806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:00.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.896+0000 7fb0b8b17700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb09806ace0 0x7fb09806d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009840 tx=0x7fb0a8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:00.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:00.899+0000 7fb0b17fa700 1 -- 172.21.15.12:0/2640337313 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb0a00541e0 con 0x7fb0b40cfea0 2023-07-08T12:39:01.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.096+0000 7fb0bad7b700 1 -- 172.21.15.12:0/2640337313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0b404ea60 con 0x7fb0b40cfea0 2023-07-08T12:39:01.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.097+0000 7fb0b17fa700 1 -- 172.21.15.12:0/2640337313 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20203 (secure 0 0 0) 0x7fb0a0057800 con 0x7fb0b40cfea0 2023-07-08T12:39:01.099 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:01.100 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:49.386462+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:39:01.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.099+0000 7fb0bad7b700 1 -- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb09806ace0 msgr2=0x7fb09806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.099+0000 7fb0bad7b700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb09806ace0 0x7fb09806d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009840 tx=0x7fb0a8008030 comp rx=0 tx=0).stop 2023-07-08T12:39:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.099+0000 7fb0bad7b700 1 -- 172.21.15.12:0/2640337313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0b40cfea0 msgr2=0x7fb0b41e1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.099+0000 7fb0bad7b700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0b40cfea0 0x7fb0b41e1310 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000b4a0 tx=0x7fb0a000b7b0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.099+0000 7fb0bad7b700 1 -- 172.21.15.12:0/2640337313 shutdown_connections 2023-07-08T12:39:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.099+0000 7fb0bad7b700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb09806ace0 0x7fb09806d190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.099+0000 7fb0bad7b700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0b415a480 0x7fb0b41df700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.099+0000 7fb0bad7b700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b40b2410 0x7fb0b4148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.099+0000 7fb0bad7b700 1 --2- 172.21.15.12:0/2640337313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0b40cfea0 0x7fb0b41e1310 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.099+0000 7fb0bad7b700 1 -- 172.21.15.12:0/2640337313 >> 172.21.15.12:0/2640337313 conn(0x7fb0b406ab60 msgr2=0x7fb0b40b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:01.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.099+0000 7fb0bad7b700 1 -- 172.21.15.12:0/2640337313 shutdown_connections 2023-07-08T12:39:01.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.099+0000 7fb0bad7b700 1 -- 172.21.15.12:0/2640337313 wait complete. 2023-07-08T12:39:01.114 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-08T12:38:05.660194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-08T12:38:06.920177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-08T12:38:10.592749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-08T12:39:01.114 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:39:01.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.240+0000 7f027f4e7700 1 Processor -- start 2023-07-08T12:39:01.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.240+0000 7f027f4e7700 1 -- start start 2023-07-08T12:39:01.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027f4e7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f027806e0c0 0x7f02781608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027f4e7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f02780cc240 0x7f02780cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027f4e7700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0278160410 con 0x7f0278156a10 2023-07-08T12:39:01.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027f4e7700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02780ccc00 con 0x7f02780cc240 2023-07-08T12:39:01.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027f4e7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0278156de0 con 0x7f027806e0c0 2023-07-08T12:39:01.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027da84700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f02780cc240 0x7f02780cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027d283700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0278156a10 0x7f027815d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38058/0 (socket says 172.21.15.12:38058) 2023-07-08T12:39:01.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027d283700 1 -- 172.21.15.12:0/2001855482 learned_addr learned my addr 172.21.15.12:0/2001855482 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:01.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027ca82700 1 --2- 172.21.15.12:0/2001855482 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f027806e0c0 0x7f02781608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027da84700 1 -- 172.21.15.12:0/2001855482 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f027806e0c0 msgr2=0x7f02781608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027da84700 1 --2- 172.21.15.12:0/2001855482 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f027806e0c0 0x7f02781608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027da84700 1 -- 172.21.15.12:0/2001855482 >> v1:172.21.15.12:6789/0 conn(0x7f0278156a10 legacy=0x7f027815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027da84700 1 -- 172.21.15.12:0/2001855482 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027806ec70 con 0x7f02780cc240 2023-07-08T12:39:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f026ffff700 1 -- 172.21.15.12:0/2001855482 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903352304 0 0) 0x7f0278160410 con 0x7f0278156a10 2023-07-08T12:39:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.241+0000 7f027ca82700 1 --2- 172.21.15.12:0/2001855482 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f027806e0c0 0x7f02781608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.242+0000 7f027da84700 1 --2- 172.21.15.12:0/2001855482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780cc240 0x7f02780cc6c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0270009d90 tx=0x7f027000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397c2f8faa3d2de4 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.242+0000 7f026ffff700 1 -- 172.21.15.12:0/2001855482 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027000e070 con 0x7f02780cc240 2023-07-08T12:39:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.242+0000 7f026ffff700 1 -- 172.21.15.12:0/2001855482 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027000bbd0 con 0x7f02780cc240 2023-07-08T12:39:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.242+0000 7f026ffff700 1 -- 172.21.15.12:0/2001855482 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270012430 con 0x7f02780cc240 2023-07-08T12:39:01.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.242+0000 7f027f4e7700 1 -- 172.21.15.12:0/2001855482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780cc240 msgr2=0x7f02780cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.242+0000 7f027f4e7700 1 --2- 172.21.15.12:0/2001855482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780cc240 0x7f02780cc6c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0270009d90 tx=0x7f027000b330 comp rx=0 tx=0).stop 2023-07-08T12:39:01.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.242+0000 7f027f4e7700 1 -- 172.21.15.12:0/2001855482 shutdown_connections 2023-07-08T12:39:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.242+0000 7f027f4e7700 1 --2- 172.21.15.12:0/2001855482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780cc240 0x7f02780cc6c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.242+0000 7f027f4e7700 1 --2- 172.21.15.12:0/2001855482 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f027806e0c0 0x7f02781608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.242+0000 7f027f4e7700 1 -- 172.21.15.12:0/2001855482 >> 172.21.15.12:0/2001855482 conn(0x7f0278098b50 msgr2=0x7f027806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.242+0000 7f027f4e7700 1 -- 172.21.15.12:0/2001855482 shutdown_connections 2023-07-08T12:39:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.242+0000 7f027f4e7700 1 -- 172.21.15.12:0/2001855482 wait complete. 2023-07-08T12:39:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.244+0000 7f027f4e7700 1 Processor -- start 2023-07-08T12:39:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.244+0000 7f027f4e7700 1 -- start start 2023-07-08T12:39:01.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.244+0000 7f027f4e7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f027806e0c0 0x7f0278157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.244+0000 7f027f4e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780cc240 0x7f0278157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.244+0000 7f027f4e7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0278156a10 0x7f0278157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.244+0000 7f027f4e7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02780add90 con 0x7f02780cc240 2023-07-08T12:39:01.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.244+0000 7f027f4e7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02780b2900 con 0x7f0278156a10 2023-07-08T12:39:01.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.244+0000 7f027f4e7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02780ad210 con 0x7f027806e0c0 2023-07-08T12:39:01.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.244+0000 7f027ca82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780cc240 0x7f0278157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.244+0000 7f027ca82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780cc240 0x7f0278157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50670/0 (socket says 172.21.15.12:50670) 2023-07-08T12:39:01.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.244+0000 7f027ca82700 1 -- 172.21.15.12:0/1902732860 learned_addr learned my addr 172.21.15.12:0/1902732860 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:01.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.245+0000 7f027d283700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f027806e0c0 0x7f0278157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.245+0000 7f027da84700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0278156a10 0x7f0278157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.245+0000 7f027ca82700 1 -- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f027806e0c0 msgr2=0x7f0278157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.245+0000 7f027ca82700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f027806e0c0 0x7f0278157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.245+0000 7f027ca82700 1 -- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0278156a10 msgr2=0x7f0278157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.245+0000 7f027ca82700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0278156a10 0x7f0278157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.245+0000 7f027ca82700 1 -- 172.21.15.12:0/1902732860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0270009a70 con 0x7f02780cc240 2023-07-08T12:39:01.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.245+0000 7f027ca82700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780cc240 0x7f0278157840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f027400e810 tx=0x7f027400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:01.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.245+0000 7f026dffb700 1 -- 172.21.15.12:0/1902732860 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0274004230 con 0x7f02780cc240 2023-07-08T12:39:01.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.245+0000 7f026dffb700 1 -- 172.21.15.12:0/1902732860 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0274004390 con 0x7f02780cc240 2023-07-08T12:39:01.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.245+0000 7f027f4e7700 1 -- 172.21.15.12:0/1902732860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02780ad4f0 con 0x7f02780cc240 2023-07-08T12:39:01.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.245+0000 7f026dffb700 1 -- 172.21.15.12:0/1902732860 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02740036a0 con 0x7f02780cc240 2023-07-08T12:39:01.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.245+0000 7f027f4e7700 1 -- 172.21.15.12:0/1902732860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02780adab0 con 0x7f02780cc240 2023-07-08T12:39:01.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.246+0000 7f027f4e7700 1 -- 172.21.15.12:0/1902732860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0278157150 con 0x7f02780cc240 2023-07-08T12:39:01.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.247+0000 7f026dffb700 1 -- 172.21.15.12:0/1902732860 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0274003800 con 0x7f02780cc240 2023-07-08T12:39:01.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.247+0000 7f026dffb700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f025c06ace0 0x7f025c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.247+0000 7f026dffb700 1 -- 172.21.15.12:0/1902732860 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0274089260 con 0x7f02780cc240 2023-07-08T12:39:01.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.247+0000 7f027d283700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f025c06ace0 0x7f025c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.248+0000 7f027d283700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f025c06ace0 0x7f025c06d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0268001670 tx=0x7f0268009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:01.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.251+0000 7f026dffb700 1 -- 172.21.15.12:0/1902732860 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0274054070 con 0x7f02780cc240 2023-07-08T12:39:01.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.448+0000 7f027f4e7700 1 -- 172.21.15.12:0/1902732860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f027804ea60 con 0x7f02780cc240 2023-07-08T12:39:01.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.448+0000 7f026dffb700 1 -- 172.21.15.12:0/1902732860 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0274057690 con 0x7f02780cc240 2023-07-08T12:39:01.451 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:39:01.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.450+0000 7f027f4e7700 1 -- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f025c06ace0 msgr2=0x7f025c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.450+0000 7f027f4e7700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f025c06ace0 0x7f025c06d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0268001670 tx=0x7f0268009040 comp rx=0 tx=0).stop 2023-07-08T12:39:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.450+0000 7f027f4e7700 1 -- 172.21.15.12:0/1902732860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780cc240 msgr2=0x7f0278157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.450+0000 7f027f4e7700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780cc240 0x7f0278157840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f027400e810 tx=0x7f027400ebd0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.450+0000 7f027f4e7700 1 -- 172.21.15.12:0/1902732860 shutdown_connections 2023-07-08T12:39:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.450+0000 7f027f4e7700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f025c06ace0 0x7f025c06d190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.450+0000 7f027f4e7700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0278156a10 0x7f0278157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.450+0000 7f027f4e7700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f027806e0c0 0x7f0278157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.450+0000 7f027f4e7700 1 --2- 172.21.15.12:0/1902732860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02780cc240 0x7f0278157840 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.450+0000 7f027f4e7700 1 -- 172.21.15.12:0/1902732860 >> 172.21.15.12:0/1902732860 conn(0x7f0278098b50 msgr2=0x7f02780cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.451+0000 7f027f4e7700 1 -- 172.21.15.12:0/1902732860 shutdown_connections 2023-07-08T12:39:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.451+0000 7f027f4e7700 1 -- 172.21.15.12:0/1902732860 wait complete. 2023-07-08T12:39:01.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:39:01.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.590+0000 7f85c86ab700 1 Processor -- start 2023-07-08T12:39:01.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.590+0000 7f85c86ab700 1 -- start start 2023-07-08T12:39:01.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.591+0000 7f85c86ab700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f85c006e0c0 0x7f85c01608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.591+0000 7f85c86ab700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c0160410 con 0x7f85c00cc240 2023-07-08T12:39:01.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.591+0000 7f85c86ab700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c0156de0 con 0x7f85c0156a10 2023-07-08T12:39:01.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.591+0000 7f85c86ab700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c006ec70 con 0x7f85c006e0c0 2023-07-08T12:39:01.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.591+0000 7f85c6c48700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f85c00cc240 0x7f85c00ae440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38076/0 (socket says 172.21.15.12:38076) 2023-07-08T12:39:01.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.591+0000 7f85c6c48700 1 -- 172.21.15.12:0/327631350 learned_addr learned my addr 172.21.15.12:0/327631350 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:01.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.591+0000 7f85c5c46700 1 --2- 172.21.15.12:0/327631350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f85c006e0c0 0x7f85c01608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.591+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 733030855 0 0) 0x7f85c0160410 con 0x7f85c00cc240 2023-07-08T12:39:01.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.591+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85ac0035b0 con 0x7f85c00cc240 2023-07-08T12:39:01.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2215354424 0 0) 0x7f85ac0035b0 con 0x7f85c00cc240 2023-07-08T12:39:01.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f85c006e0c0 msgr2=0x7f85c01608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:01.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 --2- 172.21.15.12:0/327631350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f85c006e0c0 0x7f85c01608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 >> v1:172.21.15.160:6789/0 conn(0x7f85c0156a10 legacy=0x7f85c015d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c006e6b0 con 0x7f85c00cc240 2023-07-08T12:39:01.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f85b80019f0 con 0x7f85c00cc240 2023-07-08T12:39:01.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 >> v1:172.21.15.12:6789/0 conn(0x7f85c00cc240 legacy=0x7f85c00ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 --2- 172.21.15.12:0/327631350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f85c006e0c0 0x7f85ac003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 --2- 172.21.15.12:0/327631350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85ac003f70 0x7f85ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 --2- 172.21.15.12:0/327631350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85ac004d90 0x7f85ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c006ec70 con 0x7f85ac003f70 2023-07-08T12:39:01.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac00ba40 con 0x7f85ac004d90 2023-07-08T12:39:01.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac00bc00 con 0x7f85c006e0c0 2023-07-08T12:39:01.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5c46700 1 --2- 172.21.15.12:0/327631350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85ac003f70 0x7f85ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c6447700 1 --2- 172.21.15.12:0/327631350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f85c006e0c0 0x7f85ac003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c6c48700 1 --2- 172.21.15.12:0/327631350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85ac004d90 0x7f85ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5c46700 1 -- 172.21.15.12:0/327631350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f85c006e0c0 msgr2=0x7f85ac003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:01.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5c46700 1 --2- 172.21.15.12:0/327631350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f85c006e0c0 0x7f85ac003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5c46700 1 -- 172.21.15.12:0/327631350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85ac004d90 msgr2=0x7f85ac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:01.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5c46700 1 --2- 172.21.15.12:0/327631350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85ac004d90 0x7f85ac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5c46700 1 -- 172.21.15.12:0/327631350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac0035b0 con 0x7f85ac003f70 2023-07-08T12:39:01.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5c46700 1 --2- 172.21.15.12:0/327631350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85ac003f70 0x7f85ac004850 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009b60 tx=0x7f85ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d890195356a07b44 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:01.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc00e6c0 con 0x7f85ac003f70 2023-07-08T12:39:01.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc00e820 con 0x7f85ac003f70 2023-07-08T12:39:01.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.592+0000 7f85c5445700 1 -- 172.21.15.12:0/327631350 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc00eb90 con 0x7f85ac003f70 2023-07-08T12:39:01.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.593+0000 7f85c86ab700 1 -- 172.21.15.12:0/327631350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85ac003f70 msgr2=0x7f85ac004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.593+0000 7f85c86ab700 1 --2- 172.21.15.12:0/327631350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85ac003f70 0x7f85ac004850 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009b60 tx=0x7f85ac005880 comp rx=0 tx=0).stop 2023-07-08T12:39:01.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.593+0000 7f85c6c48700 1 -- 172.21.15.12:0/327631350 reap_dead start 2023-07-08T12:39:01.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.593+0000 7f85c86ab700 1 -- 172.21.15.12:0/327631350 shutdown_connections 2023-07-08T12:39:01.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.593+0000 7f85c86ab700 1 -- 172.21.15.12:0/327631350 >> 172.21.15.12:0/327631350 conn(0x7f85c0098b50 msgr2=0x7f85c006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:01.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.593+0000 7f85c86ab700 1 -- 172.21.15.12:0/327631350 shutdown_connections 2023-07-08T12:39:01.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.593+0000 7f85c86ab700 1 -- 172.21.15.12:0/327631350 wait complete. 2023-07-08T12:39:01.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.594+0000 7f85c86ab700 1 Processor -- start 2023-07-08T12:39:01.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.594+0000 7f85c86ab700 1 -- start start 2023-07-08T12:39:01.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c86ab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85ac003f70 0x7f85c0155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c86ab700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f85c00b3a40 0x7f85c0159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c86ab700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85c0152680 0x7f85c015f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c86ab700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c015def0 con 0x7f85ac003f70 2023-07-08T12:39:01.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c86ab700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c0156fd0 con 0x7f85c0152680 2023-07-08T12:39:01.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c86ab700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c0152b20 con 0x7f85c00b3a40 2023-07-08T12:39:01.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c6447700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85ac003f70 0x7f85c0155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c6447700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85ac003f70 0x7f85c0155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50688/0 (socket says 172.21.15.12:50688) 2023-07-08T12:39:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c6447700 1 -- 172.21.15.12:0/4083326642 learned_addr learned my addr 172.21.15.12:0/4083326642 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c5c46700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f85c00b3a40 0x7f85c0159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c6c48700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85c0152680 0x7f85c015f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c6447700 1 -- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f85c00b3a40 msgr2=0x7f85c0159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c6447700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f85c00b3a40 0x7f85c0159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c6447700 1 -- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85c0152680 msgr2=0x7f85c015f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.595+0000 7f85c6447700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85c0152680 0x7f85c015f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.596+0000 7f85c6447700 1 -- 172.21.15.12:0/4083326642 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c015e100 con 0x7f85ac003f70 2023-07-08T12:39:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.596+0000 7f85c6447700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85ac003f70 0x7f85c0155210 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f85b000f7d0 tx=0x7f85b000fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.596+0000 7f85b6ffd700 1 -- 172.21.15.12:0/4083326642 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b00052e0 con 0x7f85ac003f70 2023-07-08T12:39:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.596+0000 7f85b6ffd700 1 -- 172.21.15.12:0/4083326642 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b0005460 con 0x7f85ac003f70 2023-07-08T12:39:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.596+0000 7f85c86ab700 1 -- 172.21.15.12:0/4083326642 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c015e3e0 con 0x7f85ac003f70 2023-07-08T12:39:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.596+0000 7f85b6ffd700 1 -- 172.21.15.12:0/4083326642 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b0003910 con 0x7f85ac003f70 2023-07-08T12:39:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.596+0000 7f85c86ab700 1 -- 172.21.15.12:0/4083326642 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c0157500 con 0x7f85ac003f70 2023-07-08T12:39:01.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.597+0000 7f85c86ab700 1 -- 172.21.15.12:0/4083326642 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c004ea60 con 0x7f85ac003f70 2023-07-08T12:39:01.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.597+0000 7f85b6ffd700 1 -- 172.21.15.12:0/4083326642 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f85b0003a70 con 0x7f85ac003f70 2023-07-08T12:39:01.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.598+0000 7f85b6ffd700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f85a406ace0 0x7f85a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.598+0000 7f85b6ffd700 1 -- 172.21.15.12:0/4083326642 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f85b0089440 con 0x7f85ac003f70 2023-07-08T12:39:01.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.598+0000 7f85c5c46700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f85a406ace0 0x7f85a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.599+0000 7f85c5c46700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f85a406ace0 0x7f85a406d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0090b0 tx=0x7f85bc01f040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:01.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.602+0000 7f85b6ffd700 1 -- 172.21.15.12:0/4083326642 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f85b0054300 con 0x7f85ac003f70 2023-07-08T12:39:01.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.798+0000 7f85c86ab700 1 -- 172.21.15.12:0/4083326642 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f85c00ccde0 con 0x7f85ac003f70 2023-07-08T12:39:01.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.799+0000 7f85b6ffd700 1 -- 172.21.15.12:0/4083326642 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f85b0057920 con 0x7f85ac003f70 2023-07-08T12:39:01.802 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:39:01.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.801+0000 7f85c86ab700 1 -- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f85a406ace0 msgr2=0x7f85a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.801+0000 7f85c86ab700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f85a406ace0 0x7f85a406d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0090b0 tx=0x7f85bc01f040 comp rx=0 tx=0).stop 2023-07-08T12:39:01.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.801+0000 7f85c86ab700 1 -- 172.21.15.12:0/4083326642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85ac003f70 msgr2=0x7f85c0155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.801+0000 7f85c86ab700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85ac003f70 0x7f85c0155210 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f85b000f7d0 tx=0x7f85b000fae0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.801+0000 7f85c86ab700 1 -- 172.21.15.12:0/4083326642 shutdown_connections 2023-07-08T12:39:01.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.801+0000 7f85c86ab700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f85a406ace0 0x7f85a406d190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.801+0000 7f85c86ab700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85c0152680 0x7f85c015f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.801+0000 7f85c86ab700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85ac003f70 0x7f85c0155210 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.801+0000 7f85c86ab700 1 --2- 172.21.15.12:0/4083326642 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f85c00b3a40 0x7f85c0159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.801+0000 7f85c86ab700 1 -- 172.21.15.12:0/4083326642 >> 172.21.15.12:0/4083326642 conn(0x7f85c0098b50 msgr2=0x7f85c00af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:01.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.801+0000 7f85c86ab700 1 -- 172.21.15.12:0/4083326642 shutdown_connections 2023-07-08T12:39:01.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.801+0000 7f85c86ab700 1 -- 172.21.15.12:0/4083326642 wait complete. 2023-07-08T12:39:01.815 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-08T12:39:01.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.940+0000 7fe6a94ba700 1 Processor -- start 2023-07-08T12:39:01.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.940+0000 7fe6a94ba700 1 -- start start 2023-07-08T12:39:01.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a94ba700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe6a40b8430 0x7fe6a40cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a94ba700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe6a40d3730 0x7fe6a40cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a94ba700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6a40cfa90 0x7fe6a415feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a94ba700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a40b8800 con 0x7fe6a40b8430 2023-07-08T12:39:01.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a94ba700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a40d0b20 con 0x7fe6a40d3730 2023-07-08T12:39:01.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a94ba700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40cfeb0 con 0x7fe6a40cfa90 2023-07-08T12:39:01.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a2ffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe6a40b8430 0x7fe6a40cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a2ffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe6a40b8430 0x7fe6a40cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50696/0 (socket says 172.21.15.12:50696) 2023-07-08T12:39:01.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a2ffd700 1 -- 172.21.15.12:0/4058292699 learned_addr learned my addr 172.21.15.12:0/4058292699 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:01.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a27fc700 1 --2- 172.21.15.12:0/4058292699 >> v2:172.21.15.160:3300/0 conn(0x7fe6a40d3730 0x7fe6a40cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a2ffd700 1 -- 172.21.15.12:0/4058292699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6a40cfa90 msgr2=0x7fe6a415feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:01.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a2ffd700 1 --2- 172.21.15.12:0/4058292699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6a40cfa90 0x7fe6a415feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a2ffd700 1 -- 172.21.15.12:0/4058292699 >> v2:172.21.15.160:3300/0 conn(0x7fe6a40d3730 msgr2=0x7fe6a40cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:01.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a2ffd700 1 --2- 172.21.15.12:0/4058292699 >> v2:172.21.15.160:3300/0 conn(0x7fe6a40d3730 0x7fe6a40cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a2ffd700 1 -- 172.21.15.12:0/4058292699 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a40b4ee0 con 0x7fe6a40b8430 2023-07-08T12:39:01.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a27fc700 1 --2- 172.21.15.12:0/4058292699 >> v2:172.21.15.160:3300/0 conn(0x7fe6a40d3730 0x7fe6a40cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:01.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.941+0000 7fe6a2ffd700 1 --2- 172.21.15.12:0/4058292699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a40b8430 0x7fe6a40cd320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4160f50 tx=0x7fe698009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841859a3a4eae6cd server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:01.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.942+0000 7fe6a1ffb700 1 -- 172.21.15.12:0/4058292699 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69800b870 con 0x7fe6a40b8430 2023-07-08T12:39:01.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.942+0000 7fe6a1ffb700 1 -- 172.21.15.12:0/4058292699 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69800b9d0 con 0x7fe6a40b8430 2023-07-08T12:39:01.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.942+0000 7fe6a1ffb700 1 -- 172.21.15.12:0/4058292699 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69800bd60 con 0x7fe6a40b8430 2023-07-08T12:39:01.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.942+0000 7fe6a94ba700 1 -- 172.21.15.12:0/4058292699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a40b8430 msgr2=0x7fe6a40cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.942+0000 7fe6a94ba700 1 --2- 172.21.15.12:0/4058292699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a40b8430 0x7fe6a40cd320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4160f50 tx=0x7fe698009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.942+0000 7fe6a94ba700 1 -- 172.21.15.12:0/4058292699 shutdown_connections 2023-07-08T12:39:01.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.942+0000 7fe6a94ba700 1 --2- 172.21.15.12:0/4058292699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6a40cfa90 0x7fe6a415feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.942+0000 7fe6a94ba700 1 --2- 172.21.15.12:0/4058292699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a40b8430 0x7fe6a40cd320 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.942+0000 7fe6a94ba700 1 --2- 172.21.15.12:0/4058292699 >> v2:172.21.15.160:3300/0 conn(0x7fe6a40d3730 0x7fe6a40cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.942+0000 7fe6a94ba700 1 -- 172.21.15.12:0/4058292699 >> 172.21.15.12:0/4058292699 conn(0x7fe6a406b3f0 msgr2=0x7fe6a406b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:01.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.942+0000 7fe6a94ba700 1 -- 172.21.15.12:0/4058292699 shutdown_connections 2023-07-08T12:39:01.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.942+0000 7fe6a94ba700 1 -- 172.21.15.12:0/4058292699 wait complete. 2023-07-08T12:39:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.944+0000 7fe6a94ba700 1 Processor -- start 2023-07-08T12:39:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.944+0000 7fe6a94ba700 1 -- start start 2023-07-08T12:39:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.944+0000 7fe6a94ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a40b8430 0x7fe6a40c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.944+0000 7fe6a94ba700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6a40cfa90 0x7fe6a40c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.944+0000 7fe6a94ba700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6a40d3730 0x7fe6a415a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.944+0000 7fe6a94ba700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40caeb0 con 0x7fe6a40b8430 2023-07-08T12:39:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.944+0000 7fe6a94ba700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40c6670 con 0x7fe6a40d3730 2023-07-08T12:39:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.944+0000 7fe6a94ba700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40ca2a0 con 0x7fe6a40cfa90 2023-07-08T12:39:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.944+0000 7fe6a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a40b8430 0x7fe6a40c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.944+0000 7fe6a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a40b8430 0x7fe6a40c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50702/0 (socket says 172.21.15.12:50702) 2023-07-08T12:39:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.944+0000 7fe6a2ffd700 1 -- 172.21.15.12:0/553845246 learned_addr learned my addr 172.21.15.12:0/553845246 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:01.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.945+0000 7fe6a27fc700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6a40cfa90 0x7fe6a40c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.945+0000 7fe6a37fe700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6a40d3730 0x7fe6a415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.945+0000 7fe6a2ffd700 1 -- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6a40cfa90 msgr2=0x7fe6a40c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.945+0000 7fe6a2ffd700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6a40cfa90 0x7fe6a40c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.945+0000 7fe6a2ffd700 1 -- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6a40d3730 msgr2=0x7fe6a415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:01.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.945+0000 7fe6a2ffd700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6a40d3730 0x7fe6a415a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:01.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.945+0000 7fe6a2ffd700 1 -- 172.21.15.12:0/553845246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe698009550 con 0x7fe6a40b8430 2023-07-08T12:39:01.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.945+0000 7fe6a2ffd700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a40b8430 0x7fe6a40c9c10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe698014040 tx=0x7fe698004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:01.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.945+0000 7fe68b7fe700 1 -- 172.21.15.12:0/553845246 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698004520 con 0x7fe6a40b8430 2023-07-08T12:39:01.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.945+0000 7fe68b7fe700 1 -- 172.21.15.12:0/553845246 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69801e070 con 0x7fe6a40b8430 2023-07-08T12:39:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.945+0000 7fe6a94ba700 1 -- 172.21.15.12:0/553845246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a40ca580 con 0x7fe6a40b8430 2023-07-08T12:39:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.945+0000 7fe68b7fe700 1 -- 172.21.15.12:0/553845246 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6980038e0 con 0x7fe6a40b8430 2023-07-08T12:39:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.945+0000 7fe6a94ba700 1 -- 172.21.15.12:0/553845246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a40cab40 con 0x7fe6a40b8430 2023-07-08T12:39:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.946+0000 7fe6897fa700 1 -- 172.21.15.12:0/553845246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67c0052f0 con 0x7fe6a40b8430 2023-07-08T12:39:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.946+0000 7fe68b7fe700 1 -- 172.21.15.12:0/553845246 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe698003a40 con 0x7fe6a40b8430 2023-07-08T12:39:01.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.947+0000 7fe68b7fe700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe69006ace0 0x7fe69006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:01.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.947+0000 7fe68b7fe700 1 -- 172.21.15.12:0/553845246 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe698020070 con 0x7fe6a40b8430 2023-07-08T12:39:01.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.947+0000 7fe6a27fc700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe69006ace0 0x7fe69006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:01.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.948+0000 7fe6a27fc700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe69006ace0 0x7fe69006d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe694006fd0 tx=0x7fe694008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:01.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:01.951+0000 7fe68b7fe700 1 -- 172.21.15.12:0/553845246 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe698091080 con 0x7fe6a40b8430 2023-07-08T12:39:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.148+0000 7fe6897fa700 1 -- 172.21.15.12:0/553845246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe67c005b20 con 0x7fe6a40b8430 2023-07-08T12:39:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.149+0000 7fe68b7fe700 1 -- 172.21.15.12:0/553845246 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe698015020 con 0x7fe6a40b8430 2023-07-08T12:39:02.151 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:02.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.151+0000 7fe6897fa700 1 -- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe69006ace0 msgr2=0x7fe69006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:02.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.151+0000 7fe6897fa700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe69006ace0 0x7fe69006d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe694006fd0 tx=0x7fe694008040 comp rx=0 tx=0).stop 2023-07-08T12:39:02.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.151+0000 7fe6897fa700 1 -- 172.21.15.12:0/553845246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a40b8430 msgr2=0x7fe6a40c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:02.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.151+0000 7fe6897fa700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a40b8430 0x7fe6a40c9c10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe698014040 tx=0x7fe698004750 comp rx=0 tx=0).stop 2023-07-08T12:39:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.151+0000 7fe6897fa700 1 -- 172.21.15.12:0/553845246 shutdown_connections 2023-07-08T12:39:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.151+0000 7fe6897fa700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe69006ace0 0x7fe69006d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.151+0000 7fe6897fa700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6a40d3730 0x7fe6a415a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.151+0000 7fe6897fa700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6a40b8430 0x7fe6a40c9c10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.151+0000 7fe6897fa700 1 --2- 172.21.15.12:0/553845246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6a40cfa90 0x7fe6a40c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.151+0000 7fe6897fa700 1 -- 172.21.15.12:0/553845246 >> 172.21.15.12:0/553845246 conn(0x7fe6a406b3f0 msgr2=0x7fe6a40d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.151+0000 7fe6897fa700 1 -- 172.21.15.12:0/553845246 shutdown_connections 2023-07-08T12:39:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.151+0000 7fe6897fa700 1 -- 172.21.15.12:0/553845246 wait complete. 2023-07-08T12:39:02.166 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-08T12:39:02.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.290+0000 7f6cb381f700 1 Processor -- start 2023-07-08T12:39:02.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.291+0000 7f6cb381f700 1 -- start start 2023-07-08T12:39:02.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.291+0000 7f6cb381f700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6cac15a210 0x7f6cac0c9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:02.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.291+0000 7f6cb381f700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6cac0c9b80 0x7f6cac0cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:02.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.291+0000 7f6cb381f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac0cc3d0 0x7f6cac070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:02.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.291+0000 7f6cb381f700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cac0cb760 con 0x7f6cac15a210 2023-07-08T12:39:02.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.291+0000 7f6cb381f700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cac0cc850 con 0x7f6cac0c9b80 2023-07-08T12:39:02.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.291+0000 7f6cb381f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac069f30 con 0x7f6cac0cc3d0 2023-07-08T12:39:02.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6cb15bb700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6cac15a210 0x7f6cac0c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:02.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6cb15bb700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6cac15a210 0x7f6cac0c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50710/0 (socket says 172.21.15.12:50710) 2023-07-08T12:39:02.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6cb15bb700 1 -- 172.21.15.12:0/1381648715 learned_addr learned my addr 172.21.15.12:0/1381648715 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:02.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6cb0dba700 1 --2- 172.21.15.12:0/1381648715 >> v2:172.21.15.160:3300/0 conn(0x7f6cac0c9b80 0x7f6cac0cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6cb1dbc700 1 --2- 172.21.15.12:0/1381648715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac0cc3d0 0x7f6cac070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6cb15bb700 1 -- 172.21.15.12:0/1381648715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac0cc3d0 msgr2=0x7f6cac070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6cb15bb700 1 --2- 172.21.15.12:0/1381648715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac0cc3d0 0x7f6cac070ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6cb15bb700 1 -- 172.21.15.12:0/1381648715 >> v2:172.21.15.160:3300/0 conn(0x7f6cac0c9b80 msgr2=0x7f6cac0cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6cb15bb700 1 --2- 172.21.15.12:0/1381648715 >> v2:172.21.15.160:3300/0 conn(0x7f6cac0c9b80 0x7f6cac0cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6cb15bb700 1 -- 172.21.15.12:0/1381648715 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac06a1d0 con 0x7f6cac15a210 2023-07-08T12:39:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6cb15bb700 1 --2- 172.21.15.12:0/1381648715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15a210 0x7f6cac0c9640 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6cac06e4d0 tx=0x7f6c9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165a4982a9b7a173 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6ca3fff700 1 -- 172.21.15.12:0/1381648715 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c011830 con 0x7f6cac15a210 2023-07-08T12:39:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6ca3fff700 1 -- 172.21.15.12:0/1381648715 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c011990 con 0x7f6cac15a210 2023-07-08T12:39:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.292+0000 7f6ca3fff700 1 -- 172.21.15.12:0/1381648715 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c011d20 con 0x7f6cac15a210 2023-07-08T12:39:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.293+0000 7f6cb381f700 1 -- 172.21.15.12:0/1381648715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15a210 msgr2=0x7f6cac0c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.293+0000 7f6cb381f700 1 --2- 172.21.15.12:0/1381648715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15a210 0x7f6cac0c9640 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6cac06e4d0 tx=0x7f6c9c009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.293+0000 7f6cb381f700 1 -- 172.21.15.12:0/1381648715 shutdown_connections 2023-07-08T12:39:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.293+0000 7f6cb381f700 1 --2- 172.21.15.12:0/1381648715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac0cc3d0 0x7f6cac070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.293+0000 7f6cb381f700 1 --2- 172.21.15.12:0/1381648715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15a210 0x7f6cac0c9640 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.293+0000 7f6cb381f700 1 --2- 172.21.15.12:0/1381648715 >> v2:172.21.15.160:3300/0 conn(0x7f6cac0c9b80 0x7f6cac0cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.293+0000 7f6cb381f700 1 -- 172.21.15.12:0/1381648715 >> 172.21.15.12:0/1381648715 conn(0x7f6cac0cf1d0 msgr2=0x7f6cac0cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.293+0000 7f6cb381f700 1 -- 172.21.15.12:0/1381648715 shutdown_connections 2023-07-08T12:39:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.293+0000 7f6cb381f700 1 -- 172.21.15.12:0/1381648715 wait complete. 2023-07-08T12:39:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.295+0000 7f6cb381f700 1 Processor -- start 2023-07-08T12:39:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.295+0000 7f6cb381f700 1 -- start start 2023-07-08T12:39:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.295+0000 7f6cb381f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac0c9b80 0x7f6cac14d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.295+0000 7f6cb381f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0cc3d0 0x7f6cac0c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.295+0000 7f6cb381f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15a210 0x7f6cac0c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.295+0000 7f6cb381f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac156950 con 0x7f6cac15a210 2023-07-08T12:39:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.295+0000 7f6cb381f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac0c3c80 con 0x7f6cac0cc3d0 2023-07-08T12:39:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.295+0000 7f6cb381f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac14d920 con 0x7f6cac0c9b80 2023-07-08T12:39:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.295+0000 7f6cb1dbc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15a210 0x7f6cac0c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.295+0000 7f6cb1dbc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15a210 0x7f6cac0c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50720/0 (socket says 172.21.15.12:50720) 2023-07-08T12:39:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.295+0000 7f6cb1dbc700 1 -- 172.21.15.12:0/265963289 learned_addr learned my addr 172.21.15.12:0/265963289 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.296+0000 7f6cb0dba700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0cc3d0 0x7f6cac0c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.296+0000 7f6cb15bb700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac0c9b80 0x7f6cac14d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:02.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.296+0000 7f6cb1dbc700 1 -- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac0c9b80 msgr2=0x7f6cac14d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:02.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.296+0000 7f6cb1dbc700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac0c9b80 0x7f6cac14d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.296+0000 7f6cb1dbc700 1 -- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0cc3d0 msgr2=0x7f6cac0c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:02.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.296+0000 7f6cb1dbc700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0cc3d0 0x7f6cac0c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.296+0000 7f6cb1dbc700 1 -- 172.21.15.12:0/265963289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c009550 con 0x7f6cac15a210 2023-07-08T12:39:02.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.296+0000 7f6cb1dbc700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15a210 0x7f6cac0c6f10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400efc0 tx=0x7f6ca400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:02.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.297+0000 7f6ca1ffb700 1 -- 172.21.15.12:0/265963289 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4009ea0 con 0x7f6cac15a210 2023-07-08T12:39:02.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.297+0000 7f6ca1ffb700 1 -- 172.21.15.12:0/265963289 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca400f040 con 0x7f6cac15a210 2023-07-08T12:39:02.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.297+0000 7f6ca1ffb700 1 -- 172.21.15.12:0/265963289 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca400ced0 con 0x7f6cac15a210 2023-07-08T12:39:02.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.297+0000 7f6cb381f700 1 -- 172.21.15.12:0/265963289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac0c3f20 con 0x7f6cac15a210 2023-07-08T12:39:02.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.297+0000 7f6cb381f700 1 -- 172.21.15.12:0/265963289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac0c44e0 con 0x7f6cac15a210 2023-07-08T12:39:02.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.298+0000 7f6cb381f700 1 -- 172.21.15.12:0/265963289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cac0c4700 con 0x7f6cac15a210 2023-07-08T12:39:02.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.299+0000 7f6ca1ffb700 1 -- 172.21.15.12:0/265963289 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ca4014480 con 0x7f6cac15a210 2023-07-08T12:39:02.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.299+0000 7f6ca1ffb700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6c9006ace0 0x7f6c9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:02.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.299+0000 7f6ca1ffb700 1 -- 172.21.15.12:0/265963289 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6ca4089e70 con 0x7f6cac15a210 2023-07-08T12:39:02.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.299+0000 7f6cb15bb700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6c9006ace0 0x7f6c9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:02.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.300+0000 7f6cb15bb700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6c9006ace0 0x7f6c9006d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009c10 tx=0x7f6c9c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:02.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.303+0000 7f6ca1ffb700 1 -- 172.21.15.12:0/265963289 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6ca4054c80 con 0x7f6cac15a210 2023-07-08T12:39:02.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.502+0000 7f6cb381f700 1 -- 172.21.15.12:0/265963289 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6cac04ea60 con 0x7f6cac15a210 2023-07-08T12:39:02.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.503+0000 7f6ca1ffb700 1 -- 172.21.15.12:0/265963289 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ca40180a0 con 0x7f6cac15a210 2023-07-08T12:39:02.507 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-07-08T12:39:02.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.505+0000 7f6cb381f700 1 -- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6c9006ace0 msgr2=0x7f6c9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:02.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.505+0000 7f6cb381f700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6c9006ace0 0x7f6c9006d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009c10 tx=0x7f6c9c008030 comp rx=0 tx=0).stop 2023-07-08T12:39:02.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.505+0000 7f6cb381f700 1 -- 172.21.15.12:0/265963289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15a210 msgr2=0x7f6cac0c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:02.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.505+0000 7f6cb381f700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15a210 0x7f6cac0c6f10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400efc0 tx=0x7f6ca400c3a0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.505+0000 7f6cb381f700 1 -- 172.21.15.12:0/265963289 shutdown_connections 2023-07-08T12:39:02.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.505+0000 7f6cb381f700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6c9006ace0 0x7f6c9006d190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.505+0000 7f6cb381f700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15a210 0x7f6cac0c6f10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.505+0000 7f6cb381f700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac0c9b80 0x7f6cac14d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.505+0000 7f6cb381f700 1 --2- 172.21.15.12:0/265963289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac0cc3d0 0x7f6cac0c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.505+0000 7f6cb381f700 1 -- 172.21.15.12:0/265963289 >> 172.21.15.12:0/265963289 conn(0x7f6cac0cf1d0 msgr2=0x7f6cac159560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:02.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.505+0000 7f6cb381f700 1 -- 172.21.15.12:0/265963289 shutdown_connections 2023-07-08T12:39:02.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.505+0000 7f6cb381f700 1 -- 172.21.15.12:0/265963289 wait complete. 2023-07-08T12:39:02.520 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-08T12:39:02.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.647+0000 7f131452d700 1 Processor -- start 2023-07-08T12:39:02.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.647+0000 7f131452d700 1 -- start start 2023-07-08T12:39:02.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.648+0000 7f131452d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f130c0aeec0 0x7f130c06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:02.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.648+0000 7f131452d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f130c06ebe0 0x7f130c0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:02.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.648+0000 7f131452d700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130c15d770 con 0x7f130c0cd9b0 2023-07-08T12:39:02.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.648+0000 7f131452d700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f130c15a600 con 0x7f130c0aeec0 2023-07-08T12:39:02.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.648+0000 7f131452d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0cddd0 con 0x7f130c06ebe0 2023-07-08T12:39:02.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.648+0000 7f13122c9700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f130c0aeec0 0x7f130c06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:02.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.648+0000 7f13122c9700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f130c0aeec0 0x7f130c06cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50724/0 (socket says 172.21.15.12:50724) 2023-07-08T12:39:02.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.648+0000 7f13122c9700 1 -- 172.21.15.12:0/1650755296 learned_addr learned my addr 172.21.15.12:0/1650755296 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:02.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.648+0000 7f1311ac8700 1 --2- 172.21.15.12:0/1650755296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f130c06ebe0 0x7f130c0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:02.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.648+0000 7f13122c9700 1 -- 172.21.15.12:0/1650755296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f130c06ebe0 msgr2=0x7f130c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.648+0000 7f13122c9700 1 --2- 172.21.15.12:0/1650755296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f130c06ebe0 0x7f130c0cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.648+0000 7f13122c9700 1 -- 172.21.15.12:0/1650755296 >> v1:172.21.15.160:6789/0 conn(0x7f130c0cd9b0 legacy=0x7f130c0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.648+0000 7f13122c9700 1 -- 172.21.15.12:0/1650755296 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c15cb60 con 0x7f130c0aeec0 2023-07-08T12:39:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.649+0000 7f13122c9700 1 --2- 172.21.15.12:0/1650755296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f130c0aeec0 0x7f130c06cff0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f12fc006fd0 tx=0x7f12fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a269a43b01b7b43 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.649+0000 7f13112c7700 1 -- 172.21.15.12:0/1650755296 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc005e30 con 0x7f130c0aeec0 2023-07-08T12:39:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.649+0000 7f13112c7700 1 -- 172.21.15.12:0/1650755296 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc004030 con 0x7f130c0aeec0 2023-07-08T12:39:02.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.649+0000 7f13112c7700 1 -- 172.21.15.12:0/1650755296 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc0043e0 con 0x7f130c0aeec0 2023-07-08T12:39:02.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.649+0000 7f131452d700 1 -- 172.21.15.12:0/1650755296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f130c0aeec0 msgr2=0x7f130c06cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:02.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.649+0000 7f131452d700 1 --2- 172.21.15.12:0/1650755296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f130c0aeec0 0x7f130c06cff0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f12fc006fd0 tx=0x7f12fc009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.649+0000 7f131452d700 1 -- 172.21.15.12:0/1650755296 shutdown_connections 2023-07-08T12:39:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.649+0000 7f131452d700 1 --2- 172.21.15.12:0/1650755296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f130c0aeec0 0x7f130c06cff0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.649+0000 7f131452d700 1 --2- 172.21.15.12:0/1650755296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f130c06ebe0 0x7f130c0cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.649+0000 7f131452d700 1 -- 172.21.15.12:0/1650755296 >> 172.21.15.12:0/1650755296 conn(0x7f130c098b50 msgr2=0x7f130c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.649+0000 7f131452d700 1 -- 172.21.15.12:0/1650755296 shutdown_connections 2023-07-08T12:39:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.649+0000 7f131452d700 1 -- 172.21.15.12:0/1650755296 wait complete. 2023-07-08T12:39:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.650+0000 7f131452d700 1 Processor -- start 2023-07-08T12:39:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.651+0000 7f131452d700 1 -- start start 2023-07-08T12:39:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.651+0000 7f131452d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f130c06ebe0 0x7f130c0c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.651+0000 7f131452d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f130c0aeec0 0x7f130c0cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.651+0000 7f131452d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f130c0cd9b0 0x7f130c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.651+0000 7f131452d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c14e5f0 con 0x7f130c06ebe0 2023-07-08T12:39:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.651+0000 7f131452d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0af990 con 0x7f130c0aeec0 2023-07-08T12:39:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.651+0000 7f131452d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0afc50 con 0x7f130c0cd9b0 2023-07-08T12:39:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.651+0000 7f13122c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f130c06ebe0 0x7f130c0c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:02.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.651+0000 7f13122c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f130c06ebe0 0x7f130c0c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50730/0 (socket says 172.21.15.12:50730) 2023-07-08T12:39:02.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.651+0000 7f13122c9700 1 -- 172.21.15.12:0/3743459615 learned_addr learned my addr 172.21.15.12:0/3743459615 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:02.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.652+0000 7f1311ac8700 1 --2- 172.21.15.12:0/3743459615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f130c0aeec0 0x7f130c0cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:02.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.652+0000 7f13122c9700 1 -- 172.21.15.12:0/3743459615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f130c0cd9b0 msgr2=0x7f130c069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:02.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.652+0000 7f13122c9700 1 --2- 172.21.15.12:0/3743459615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f130c0cd9b0 0x7f130c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.652+0000 7f13122c9700 1 -- 172.21.15.12:0/3743459615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f130c0aeec0 msgr2=0x7f130c0cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:02.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.652+0000 7f13122c9700 1 --2- 172.21.15.12:0/3743459615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f130c0aeec0 0x7f130c0cbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.652+0000 7f13122c9700 1 -- 172.21.15.12:0/3743459615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc009500 con 0x7f130c06ebe0 2023-07-08T12:39:02.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.652+0000 7f13122c9700 1 --2- 172.21.15.12:0/3743459615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f130c06ebe0 0x7f130c0c8730 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009c10 tx=0x7f12fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:02.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.652+0000 7f1302ffd700 1 -- 172.21.15.12:0/3743459615 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc003dd0 con 0x7f130c06ebe0 2023-07-08T12:39:02.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.652+0000 7f1302ffd700 1 -- 172.21.15.12:0/3743459615 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc01c420 con 0x7f130c06ebe0 2023-07-08T12:39:02.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.652+0000 7f131452d700 1 -- 172.21.15.12:0/3743459615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130c0cabc0 con 0x7f130c06ebe0 2023-07-08T12:39:02.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.652+0000 7f1302ffd700 1 -- 172.21.15.12:0/3743459615 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc01c7b0 con 0x7f130c06ebe0 2023-07-08T12:39:02.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.652+0000 7f131452d700 1 -- 172.21.15.12:0/3743459615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c0ca600 con 0x7f130c06ebe0 2023-07-08T12:39:02.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.653+0000 7f131452d700 1 -- 172.21.15.12:0/3743459615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130c04ea60 con 0x7f130c06ebe0 2023-07-08T12:39:02.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.653+0000 7f1302ffd700 1 -- 172.21.15.12:0/3743459615 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f12fc01c910 con 0x7f130c06ebe0 2023-07-08T12:39:02.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.654+0000 7f1302ffd700 1 --2- 172.21.15.12:0/3743459615 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f12f006ace0 0x7f12f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:02.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.654+0000 7f1302ffd700 1 -- 172.21.15.12:0/3743459615 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f12fc08a820 con 0x7f130c06ebe0 2023-07-08T12:39:02.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.654+0000 7f1311ac8700 1 --2- 172.21.15.12:0/3743459615 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f12f006ace0 0x7f12f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:02.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.654+0000 7f1311ac8700 1 --2- 172.21.15.12:0/3743459615 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f12f006ace0 0x7f12f006d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f130c06ab40 tx=0x7f1308008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:02.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.658+0000 7f1302ffd700 1 -- 172.21.15.12:0/3743459615 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f12fc055630 con 0x7f130c06ebe0 2023-07-08T12:39:02.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.854+0000 7f131452d700 1 -- 172.21.15.12:0/3743459615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f130c066af0 con 0x7f130c06ebe0 2023-07-08T12:39:02.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.854+0000 7f1302ffd700 1 -- 172.21.15.12:0/3743459615 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f12fc058c50 con 0x7f130c06ebe0 2023-07-08T12:39:02.857 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:02.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.856+0000 7f131452d700 1 -- 172.21.15.12:0/3743459615 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f12f006ace0 msgr2=0x7f12f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:02.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.856+0000 7f131452d700 1 --2- 172.21.15.12:0/3743459615 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f12f006ace0 0x7f12f006d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f130c06ab40 tx=0x7f1308008040 comp rx=0 tx=0).stop 2023-07-08T12:39:02.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.856+0000 7f131452d700 1 -- 172.21.15.12:0/3743459615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f130c06ebe0 msgr2=0x7f130c0c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:02.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.856+0000 7f131452d700 1 --2- 172.21.15.12:0/3743459615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f130c06ebe0 0x7f130c0c8730 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009c10 tx=0x7f12fc004750 comp rx=0 tx=0).stop 2023-07-08T12:39:02.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.856+0000 7f131452d700 1 -- 172.21.15.12:0/3743459615 shutdown_connections 2023-07-08T12:39:02.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.856+0000 7f131452d700 1 --2- 172.21.15.12:0/3743459615 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f12f006ace0 0x7f12f006d190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.856+0000 7f131452d700 1 --2- 172.21.15.12:0/3743459615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f130c0cd9b0 0x7f130c069f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.856+0000 7f131452d700 1 --2- 172.21.15.12:0/3743459615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f130c06ebe0 0x7f130c0c8730 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.856+0000 7f131452d700 1 --2- 172.21.15.12:0/3743459615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f130c0aeec0 0x7f130c0cbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:02.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.856+0000 7f131452d700 1 -- 172.21.15.12:0/3743459615 >> 172.21.15.12:0/3743459615 conn(0x7f130c098b50 msgr2=0x7f130c0ac420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:02.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.856+0000 7f131452d700 1 -- 172.21.15.12:0/3743459615 shutdown_connections 2023-07-08T12:39:02.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:02.856+0000 7f131452d700 1 -- 172.21.15.12:0/3743459615 wait complete. 2023-07-08T12:39:02.878 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-07-08T12:39:02.879 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2023-07-08T12:39:03.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.002+0000 7f3daceb1700 1 Processor -- start 2023-07-08T12:39:03.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.002+0000 7f3daceb1700 1 -- start start 2023-07-08T12:39:03.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3daceb1700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3da815a370 0x7f3da815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:03.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3daceb1700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3da8069f30 0x7f3da806a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:03.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3daceb1700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da80b1de0 con 0x7f3da806cd50 2023-07-08T12:39:03.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3daceb1700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da815a740 con 0x7f3da815a370 2023-07-08T12:39:03.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3daceb1700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da806a8e0 con 0x7f3da8069f30 2023-07-08T12:39:03.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3da659c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3da815a370 0x7f3da815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:03.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3da6d9d700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3da806cd50 0x7f3da806d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38138/0 (socket says 172.21.15.12:38138) 2023-07-08T12:39:03.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3da6d9d700 1 -- 172.21.15.12:0/1061856017 learned_addr learned my addr 172.21.15.12:0/1061856017 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:03.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3da659c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3da815a370 0x7f3da815e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50738/0 (socket says 172.21.15.12:50738) 2023-07-08T12:39:03.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3da5d9b700 1 --2- 172.21.15.12:0/1061856017 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3da8069f30 0x7f3da806a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:03.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3da659c700 1 -- 172.21.15.12:0/1061856017 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3da8069f30 msgr2=0x7f3da806a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3da659c700 1 --2- 172.21.15.12:0/1061856017 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3da8069f30 0x7f3da806a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3da659c700 1 -- 172.21.15.12:0/1061856017 >> v1:172.21.15.12:6789/0 conn(0x7f3da806cd50 legacy=0x7f3da806d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3da659c700 1 -- 172.21.15.12:0/1061856017 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da815ed90 con 0x7f3da815a370 2023-07-08T12:39:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.003+0000 7f3da559a700 1 -- 172.21.15.12:0/1061856017 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305904973 0 0) 0x7f3da80b1de0 con 0x7f3da806cd50 2023-07-08T12:39:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.004+0000 7f3da659c700 1 --2- 172.21.15.12:0/1061856017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da815a370 0x7f3da815e850 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009870 tx=0x7f3d98009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd0e4a7cf652348 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.004+0000 7f3da559a700 1 -- 172.21.15.12:0/1061856017 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98011830 con 0x7f3da815a370 2023-07-08T12:39:03.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.004+0000 7f3da559a700 1 -- 172.21.15.12:0/1061856017 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d98011990 con 0x7f3da815a370 2023-07-08T12:39:03.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.004+0000 7f3da559a700 1 -- 172.21.15.12:0/1061856017 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98011d20 con 0x7f3da815a370 2023-07-08T12:39:03.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.004+0000 7f3daceb1700 1 -- 172.21.15.12:0/1061856017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da815a370 msgr2=0x7f3da815e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:03.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.004+0000 7f3daceb1700 1 --2- 172.21.15.12:0/1061856017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da815a370 0x7f3da815e850 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009870 tx=0x7f3d98009b80 comp rx=0 tx=0).stop 2023-07-08T12:39:03.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.004+0000 7f3daceb1700 1 -- 172.21.15.12:0/1061856017 shutdown_connections 2023-07-08T12:39:03.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.004+0000 7f3daceb1700 1 --2- 172.21.15.12:0/1061856017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da815a370 0x7f3da815e850 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.004+0000 7f3daceb1700 1 --2- 172.21.15.12:0/1061856017 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3da8069f30 0x7f3da806a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.004+0000 7f3daceb1700 1 -- 172.21.15.12:0/1061856017 >> 172.21.15.12:0/1061856017 conn(0x7f3da80cb9f0 msgr2=0x7f3da80cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:03.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.004+0000 7f3daceb1700 1 -- 172.21.15.12:0/1061856017 shutdown_connections 2023-07-08T12:39:03.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.004+0000 7f3daceb1700 1 -- 172.21.15.12:0/1061856017 wait complete. 2023-07-08T12:39:03.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.006+0000 7f3daceb1700 1 Processor -- start 2023-07-08T12:39:03.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.006+0000 7f3daceb1700 1 -- start start 2023-07-08T12:39:03.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.006+0000 7f3daceb1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3da8069f30 0x7f3da8149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:03.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.006+0000 7f3daceb1700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3da806cd50 0x7f3da80cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:03.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.006+0000 7f3daceb1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da815a370 0x7f3da80cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:03.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.006+0000 7f3daceb1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80c4150 con 0x7f3da815a370 2023-07-08T12:39:03.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.006+0000 7f3daceb1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da814aab0 con 0x7f3da8069f30 2023-07-08T12:39:03.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.006+0000 7f3daceb1700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80ce520 con 0x7f3da806cd50 2023-07-08T12:39:03.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.006+0000 7f3da6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da815a370 0x7f3da80cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:03.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3da6d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da815a370 0x7f3da80cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50748/0 (socket says 172.21.15.12:50748) 2023-07-08T12:39:03.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3da6d9d700 1 -- 172.21.15.12:0/1252420447 learned_addr learned my addr 172.21.15.12:0/1252420447 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:03.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3da5d9b700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3da806cd50 0x7f3da80cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:03.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3da659c700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3da8069f30 0x7f3da8149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:03.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3da6d9d700 1 -- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3da806cd50 msgr2=0x7f3da80cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:03.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3da6d9d700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3da806cd50 0x7f3da80cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3da6d9d700 1 -- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3da8069f30 msgr2=0x7f3da8149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:03.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3da6d9d700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3da8069f30 0x7f3da8149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3da6d9d700 1 -- 172.21.15.12:0/1252420447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d980094a0 con 0x7f3da815a370 2023-07-08T12:39:03.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3da659c700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3da8069f30 0x7f3da8149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:03.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3da5d9b700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3da806cd50 0x7f3da80cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:03.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3da6d9d700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da815a370 0x7f3da80cdef0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3da0002230 tx=0x7f3da000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:03.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3d96ffd700 1 -- 172.21.15.12:0/1252420447 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da000ac20 con 0x7f3da815a370 2023-07-08T12:39:03.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3d96ffd700 1 -- 172.21.15.12:0/1252420447 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da000ada0 con 0x7f3da815a370 2023-07-08T12:39:03.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3daceb1700 1 -- 172.21.15.12:0/1252420447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da80ce800 con 0x7f3da815a370 2023-07-08T12:39:03.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3d96ffd700 1 -- 172.21.15.12:0/1252420447 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da0007310 con 0x7f3da815a370 2023-07-08T12:39:03.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.007+0000 7f3daceb1700 1 -- 172.21.15.12:0/1252420447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da80c3de0 con 0x7f3da815a370 2023-07-08T12:39:03.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.008+0000 7f3daceb1700 1 -- 172.21.15.12:0/1252420447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da80c42c0 con 0x7f3da815a370 2023-07-08T12:39:03.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.009+0000 7f3d96ffd700 1 -- 172.21.15.12:0/1252420447 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3da0007470 con 0x7f3da815a370 2023-07-08T12:39:03.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.009+0000 7f3d96ffd700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3d8806ace0 0x7f3d8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:03.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.009+0000 7f3d96ffd700 1 -- 172.21.15.12:0/1252420447 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3da00897c0 con 0x7f3da815a370 2023-07-08T12:39:03.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.009+0000 7f3da659c700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3d8806ace0 0x7f3d8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:03.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.010+0000 7f3da659c700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3d8806ace0 0x7f3d8806d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3d98000c00 tx=0x7f3d98008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:03.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.013+0000 7f3d96ffd700 1 -- 172.21.15.12:0/1252420447 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3da00544b0 con 0x7f3da815a370 2023-07-08T12:39:03.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.289+0000 7f3daceb1700 1 -- 172.21.15.12:0/1252420447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) v1 -- 0x7f3da804ea60 con 0x7f3da815a370 2023-07-08T12:39:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.435+0000 7f3d96ffd700 1 -- 172.21.15.12:0/1252420447 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v72) v1 ==== 147+0+0 (secure 0 0 0) 0x7f3da00180d0 con 0x7f3da815a370 2023-07-08T12:39:03.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.438+0000 7f3daceb1700 1 -- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3d8806ace0 msgr2=0x7f3d8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:03.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.438+0000 7f3daceb1700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3d8806ace0 0x7f3d8806d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3d98000c00 tx=0x7f3d98008030 comp rx=0 tx=0).stop 2023-07-08T12:39:03.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.438+0000 7f3daceb1700 1 -- 172.21.15.12:0/1252420447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da815a370 msgr2=0x7f3da80cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:03.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.438+0000 7f3daceb1700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da815a370 0x7f3da80cdef0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3da0002230 tx=0x7f3da000a060 comp rx=0 tx=0).stop 2023-07-08T12:39:03.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.438+0000 7f3daceb1700 1 -- 172.21.15.12:0/1252420447 shutdown_connections 2023-07-08T12:39:03.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.438+0000 7f3daceb1700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3d8806ace0 0x7f3d8806d190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.438+0000 7f3daceb1700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3da815a370 0x7f3da80cdef0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.438+0000 7f3daceb1700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3da8069f30 0x7f3da8149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.438+0000 7f3daceb1700 1 --2- 172.21.15.12:0/1252420447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3da806cd50 0x7f3da80cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.438+0000 7f3daceb1700 1 -- 172.21.15.12:0/1252420447 >> 172.21.15.12:0/1252420447 conn(0x7f3da80cb9f0 msgr2=0x7f3da80c7830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:03.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.438+0000 7f3daceb1700 1 -- 172.21.15.12:0/1252420447 shutdown_connections 2023-07-08T12:39:03.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.438+0000 7f3daceb1700 1 -- 172.21.15.12:0/1252420447 wait complete. 2023-07-08T12:39:03.453 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-08T12:39:03.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.580+0000 7f1dd64a1700 1 Processor -- start 2023-07-08T12:39:03.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.581+0000 7f1dd64a1700 1 -- start start 2023-07-08T12:39:03.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.581+0000 7f1dd64a1700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dd00ce560 0x7f1dd00ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:03.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.581+0000 7f1dd64a1700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1dd015a6d0 0x7f1dd015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:03.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.581+0000 7f1dd64a1700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd0158130 con 0x7f1dd006f330 2023-07-08T12:39:03.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.581+0000 7f1dd64a1700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd015b730 con 0x7f1dd015a6d0 2023-07-08T12:39:03.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.581+0000 7f1dd64a1700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd00cee70 con 0x7f1dd00ce560 2023-07-08T12:39:03.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.582+0000 7f1dcf7fe700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1dd015a6d0 0x7f1dd015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:03.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.582+0000 7f1dcffff700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dd00ce560 0x7f1dd00ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:03.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.582+0000 7f1dcf7fe700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1dd015a6d0 0x7f1dd015ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49498/0 (socket says 172.21.15.12:49498) 2023-07-08T12:39:03.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.582+0000 7f1dcf7fe700 1 -- 172.21.15.12:0/860657378 learned_addr learned my addr 172.21.15.12:0/860657378 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:03.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.582+0000 7f1dceffd700 1 -- 172.21.15.12:0/860657378 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014148506 0 0) 0x7f1dd0158130 con 0x7f1dd006f330 2023-07-08T12:39:03.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.582+0000 7f1dceffd700 1 -- 172.21.15.12:0/860657378 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1db80035b0 con 0x7f1dd006f330 2023-07-08T12:39:03.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.582+0000 7f1dcf7fe700 1 -- 172.21.15.12:0/860657378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dd00ce560 msgr2=0x7f1dd00ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:03.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.582+0000 7f1dcf7fe700 1 --2- 172.21.15.12:0/860657378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dd00ce560 0x7f1dd00ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.582+0000 7f1dcf7fe700 1 -- 172.21.15.12:0/860657378 >> v1:172.21.15.160:6789/0 conn(0x7f1dd006f330 legacy=0x7f1dd006f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:03.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.582+0000 7f1dcf7fe700 1 -- 172.21.15.12:0/860657378 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd015b450 con 0x7f1dd015a6d0 2023-07-08T12:39:03.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.582+0000 7f1dcffff700 1 --2- 172.21.15.12:0/860657378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dd00ce560 0x7f1dd00ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:39:03.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.582+0000 7f1dcf7fe700 1 --2- 172.21.15.12:0/860657378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dd015a6d0 0x7f1dd015ab40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40098f0 tx=0x7f1dc4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=862dd0a9bb0762ac server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:03.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.583+0000 7f1dceffd700 1 -- 172.21.15.12:0/860657378 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4011830 con 0x7f1dd015a6d0 2023-07-08T12:39:03.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.583+0000 7f1dceffd700 1 -- 172.21.15.12:0/860657378 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc4011990 con 0x7f1dd015a6d0 2023-07-08T12:39:03.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.583+0000 7f1dceffd700 1 -- 172.21.15.12:0/860657378 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4011d00 con 0x7f1dd015a6d0 2023-07-08T12:39:03.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.583+0000 7f1dd64a1700 1 -- 172.21.15.12:0/860657378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dd015a6d0 msgr2=0x7f1dd015ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:03.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.583+0000 7f1dd64a1700 1 --2- 172.21.15.12:0/860657378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dd015a6d0 0x7f1dd015ab40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40098f0 tx=0x7f1dc4009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:03.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.583+0000 7f1dd64a1700 1 -- 172.21.15.12:0/860657378 shutdown_connections 2023-07-08T12:39:03.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.583+0000 7f1dd64a1700 1 --2- 172.21.15.12:0/860657378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dd00ce560 0x7f1dd00ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.583+0000 7f1dd64a1700 1 --2- 172.21.15.12:0/860657378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dd015a6d0 0x7f1dd015ab40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.583+0000 7f1dd64a1700 1 -- 172.21.15.12:0/860657378 >> 172.21.15.12:0/860657378 conn(0x7f1dd0099c60 msgr2=0x7f1dd0089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:03.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.583+0000 7f1dd64a1700 1 -- 172.21.15.12:0/860657378 shutdown_connections 2023-07-08T12:39:03.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.583+0000 7f1dd64a1700 1 -- 172.21.15.12:0/860657378 wait complete. 2023-07-08T12:39:03.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.584+0000 7f1dd64a1700 1 Processor -- start 2023-07-08T12:39:03.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.585+0000 7f1dd64a1700 1 -- start start 2023-07-08T12:39:03.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.585+0000 7f1dd64a1700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dd006f330 0x7f1dd00ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:03.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.585+0000 7f1dd64a1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dd00ce560 0x7f1dd006b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:03.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.585+0000 7f1dd64a1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dd015a6d0 0x7f1dd014d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:03.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.585+0000 7f1dd64a1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd006ba90 con 0x7f1dd015a6d0 2023-07-08T12:39:03.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.585+0000 7f1dd64a1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd006bd70 con 0x7f1dd00ce560 2023-07-08T12:39:03.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.585+0000 7f1dd64a1700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd006c050 con 0x7f1dd006f330 2023-07-08T12:39:03.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.585+0000 7f1dd4a3e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dd015a6d0 0x7f1dd014d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:03.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.585+0000 7f1dd4a3e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dd015a6d0 0x7f1dd014d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50762/0 (socket says 172.21.15.12:50762) 2023-07-08T12:39:03.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.585+0000 7f1dd4a3e700 1 -- 172.21.15.12:0/3918616486 learned_addr learned my addr 172.21.15.12:0/3918616486 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:03.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.585+0000 7f1dcffff700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dd006f330 0x7f1dd00ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:03.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.586+0000 7f1dcf7fe700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dd00ce560 0x7f1dd006b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:03.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.586+0000 7f1dd4a3e700 1 -- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dd006f330 msgr2=0x7f1dd00ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:03.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.586+0000 7f1dd4a3e700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dd006f330 0x7f1dd00ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.586+0000 7f1dd4a3e700 1 -- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dd00ce560 msgr2=0x7f1dd006b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:03.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.586+0000 7f1dd4a3e700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dd00ce560 0x7f1dd006b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.586+0000 7f1dd4a3e700 1 -- 172.21.15.12:0/3918616486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc00094a0 con 0x7f1dd015a6d0 2023-07-08T12:39:03.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.586+0000 7f1dd4a3e700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dd015a6d0 0x7f1dd014d5b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800cbc0 tx=0x7f1dc800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:03.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.586+0000 7f1dccff9700 1 -- 172.21.15.12:0/3918616486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8003c00 con 0x7f1dd015a6d0 2023-07-08T12:39:03.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.586+0000 7f1dccff9700 1 -- 172.21.15.12:0/3918616486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc8003d60 con 0x7f1dd015a6d0 2023-07-08T12:39:03.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.586+0000 7f1dd64a1700 1 -- 172.21.15.12:0/3918616486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc4009550 con 0x7f1dd015a6d0 2023-07-08T12:39:03.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.586+0000 7f1dccff9700 1 -- 172.21.15.12:0/3918616486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc800aaa0 con 0x7f1dd015a6d0 2023-07-08T12:39:03.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.586+0000 7f1dd64a1700 1 -- 172.21.15.12:0/3918616486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd006c610 con 0x7f1dd015a6d0 2023-07-08T12:39:03.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.587+0000 7f1dd64a1700 1 -- 172.21.15.12:0/3918616486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd004ea60 con 0x7f1dd015a6d0 2023-07-08T12:39:03.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.588+0000 7f1dccff9700 1 -- 172.21.15.12:0/3918616486 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1dc800ac00 con 0x7f1dd015a6d0 2023-07-08T12:39:03.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.588+0000 7f1dccff9700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1dbc06ace0 0x7f1dbc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:03.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.588+0000 7f1dccff9700 1 -- 172.21.15.12:0/3918616486 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1dc80896e0 con 0x7f1dd015a6d0 2023-07-08T12:39:03.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.588+0000 7f1dcffff700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1dbc06ace0 0x7f1dbc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:03.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.589+0000 7f1dcffff700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1dbc06ace0 0x7f1dbc06d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009cd0 tx=0x7f1dc0009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:03.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.593+0000 7f1dccff9700 1 -- 172.21.15.12:0/3918616486 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1dc80544f0 con 0x7f1dd015a6d0 2023-07-08T12:39:03.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.791+0000 7f1dd64a1700 1 -- 172.21.15.12:0/3918616486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1dd01554e0 con 0x7f1dd015a6d0 2023-07-08T12:39:03.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.797+0000 7f1dccff9700 1 -- 172.21.15.12:0/3918616486 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1dc8057b10 con 0x7f1dd015a6d0 2023-07-08T12:39:03.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.799+0000 7f1dd64a1700 1 -- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1dbc06ace0 msgr2=0x7f1dbc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:03.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.799+0000 7f1dd64a1700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1dbc06ace0 0x7f1dbc06d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009cd0 tx=0x7f1dc0009210 comp rx=0 tx=0).stop 2023-07-08T12:39:03.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.799+0000 7f1dd64a1700 1 -- 172.21.15.12:0/3918616486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dd015a6d0 msgr2=0x7f1dd014d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:03.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.799+0000 7f1dd64a1700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dd015a6d0 0x7f1dd014d5b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800cbc0 tx=0x7f1dc800ced0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.799+0000 7f1dd64a1700 1 -- 172.21.15.12:0/3918616486 shutdown_connections 2023-07-08T12:39:03.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.799+0000 7f1dd64a1700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f1dbc06ace0 0x7f1dbc06d190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.799+0000 7f1dd64a1700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dd015a6d0 0x7f1dd014d5b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.799+0000 7f1dd64a1700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dd006f330 0x7f1dd00ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.799+0000 7f1dd64a1700 1 --2- 172.21.15.12:0/3918616486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dd00ce560 0x7f1dd006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.799+0000 7f1dd64a1700 1 -- 172.21.15.12:0/3918616486 >> 172.21.15.12:0/3918616486 conn(0x7f1dd0099c60 msgr2=0x7f1dd0072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.799+0000 7f1dd64a1700 1 -- 172.21.15.12:0/3918616486 shutdown_connections 2023-07-08T12:39:03.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:03.799+0000 7f1dd64a1700 1 -- 172.21.15.12:0/3918616486 wait complete. 2023-07-08T12:39:03.814 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-08T12:39:03.814 DEBUG:teuthology.orchestra.run.smithi012:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-08T12:39:03.854 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-08T12:39:03.907 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-08T12:39:03.908 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:39:03.908 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:39:03.925 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:39:03.926 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:39:03.982 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:39:03.982 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:39:04.039 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:39:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.201+0000 7fa438b4a700 1 Processor -- start 2023-07-08T12:39:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.201+0000 7fa438b4a700 1 -- start start 2023-07-08T12:39:04.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.202+0000 7fa438b4a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa434159c30 0x7fa4340cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.202+0000 7fa438b4a700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa434160790 0x7fa43406ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.202+0000 7fa438b4a700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa43415c890 con 0x7fa434069f30 2023-07-08T12:39:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.202+0000 7fa438b4a700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43415c730 con 0x7fa434160790 2023-07-08T12:39:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.202+0000 7fa438b4a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340cf590 con 0x7fa434159c30 2023-07-08T12:39:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.203+0000 7fa43259c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa434159c30 0x7fa4340cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.203+0000 7fa431d9b700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa434160790 0x7fa43406ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.203+0000 7fa43259c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa434159c30 0x7fa4340cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:60384/0 (socket says 172.21.15.12:60384) 2023-07-08T12:39:04.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.203+0000 7fa43259c700 1 -- 172.21.15.12:0/1733586972 learned_addr learned my addr 172.21.15.12:0/1733586972 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:04.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.203+0000 7fa43259c700 1 -- 172.21.15.12:0/1733586972 >> v2:172.21.15.160:3300/0 conn(0x7fa434160790 msgr2=0x7fa43406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:04.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.203+0000 7fa43259c700 1 --2- 172.21.15.12:0/1733586972 >> v2:172.21.15.160:3300/0 conn(0x7fa434160790 0x7fa43406ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.203+0000 7fa43259c700 1 -- 172.21.15.12:0/1733586972 >> v1:172.21.15.160:6789/0 conn(0x7fa434069f30 legacy=0x7fa43415cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.203+0000 7fa43259c700 1 -- 172.21.15.12:0/1733586972 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa434160c90 con 0x7fa434159c30 2023-07-08T12:39:04.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.203+0000 7fa43159a700 1 -- 172.21.15.12:0/1733586972 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669976188 0 0) 0x7fa43415c890 con 0x7fa434069f30 2023-07-08T12:39:04.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.203+0000 7fa431d9b700 1 --2- 172.21.15.12:0/1733586972 >> v2:172.21.15.160:3300/0 conn(0x7fa434160790 0x7fa43406ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:39:04.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.203+0000 7fa43259c700 1 --2- 172.21.15.12:0/1733586972 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa434159c30 0x7fa4340cf050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4240098f0 tx=0x7fa424009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=518905dcaf9ce178 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:04.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.204+0000 7fa43159a700 1 -- 172.21.15.12:0/1733586972 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424011830 con 0x7fa434159c30 2023-07-08T12:39:04.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.204+0000 7fa43159a700 1 -- 172.21.15.12:0/1733586972 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa424011990 con 0x7fa434159c30 2023-07-08T12:39:04.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.204+0000 7fa43159a700 1 -- 172.21.15.12:0/1733586972 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424011d20 con 0x7fa434159c30 2023-07-08T12:39:04.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.204+0000 7fa438b4a700 1 -- 172.21.15.12:0/1733586972 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa434159c30 msgr2=0x7fa4340cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.204+0000 7fa438b4a700 1 --2- 172.21.15.12:0/1733586972 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa434159c30 0x7fa4340cf050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4240098f0 tx=0x7fa424009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:04.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.204+0000 7fa438b4a700 1 -- 172.21.15.12:0/1733586972 shutdown_connections 2023-07-08T12:39:04.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.204+0000 7fa438b4a700 1 --2- 172.21.15.12:0/1733586972 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa434159c30 0x7fa4340cf050 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.204+0000 7fa438b4a700 1 --2- 172.21.15.12:0/1733586972 >> v2:172.21.15.160:3300/0 conn(0x7fa434160790 0x7fa43406ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.204+0000 7fa438b4a700 1 -- 172.21.15.12:0/1733586972 >> 172.21.15.12:0/1733586972 conn(0x7fa4340cc290 msgr2=0x7fa4340cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:04.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.204+0000 7fa438b4a700 1 -- 172.21.15.12:0/1733586972 shutdown_connections 2023-07-08T12:39:04.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.204+0000 7fa438b4a700 1 -- 172.21.15.12:0/1733586972 wait complete. 2023-07-08T12:39:04.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.206+0000 7fa438b4a700 1 Processor -- start 2023-07-08T12:39:04.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.206+0000 7fa438b4a700 1 -- start start 2023-07-08T12:39:04.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.206+0000 7fa438b4a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa434069f30 0x7fa434151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.206+0000 7fa438b4a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa434159c30 0x7fa4340b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.206+0000 7fa438b4a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa434160790 0x7fa4340cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.206+0000 7fa438b4a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4341e8300 con 0x7fa434069f30 2023-07-08T12:39:04.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.206+0000 7fa438b4a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340b4770 con 0x7fa434160790 2023-07-08T12:39:04.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.206+0000 7fa438b4a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4341e7960 con 0x7fa434159c30 2023-07-08T12:39:04.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.206+0000 7fa43259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa434069f30 0x7fa434151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.206+0000 7fa43259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa434069f30 0x7fa434151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50772/0 (socket says 172.21.15.12:50772) 2023-07-08T12:39:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.206+0000 7fa43259c700 1 -- 172.21.15.12:0/2618885305 learned_addr learned my addr 172.21.15.12:0/2618885305 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa431d9b700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa434159c30 0x7fa4340b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa432d9d700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa434160790 0x7fa4340cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa43259c700 1 -- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa434159c30 msgr2=0x7fa4340b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa43259c700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa434159c30 0x7fa4340b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa43259c700 1 -- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa434160790 msgr2=0x7fa4340cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa43259c700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa434160790 0x7fa4340cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa43259c700 1 -- 172.21.15.12:0/2618885305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4280094a0 con 0x7fa434069f30 2023-07-08T12:39:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa431d9b700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa434159c30 0x7fa4340b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa432d9d700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa434160790 0x7fa4340cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa43259c700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa434069f30 0x7fa434151280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa424000c00 tx=0x7fa424004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa422ffd700 1 -- 172.21.15.12:0/2618885305 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42401ab00 con 0x7fa434069f30 2023-07-08T12:39:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa422ffd700 1 -- 172.21.15.12:0/2618885305 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42401f070 con 0x7fa434069f30 2023-07-08T12:39:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa438b4a700 1 -- 172.21.15.12:0/2618885305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa424009550 con 0x7fa434069f30 2023-07-08T12:39:04.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa422ffd700 1 -- 172.21.15.12:0/2618885305 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42401ae80 con 0x7fa434069f30 2023-07-08T12:39:04.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.207+0000 7fa438b4a700 1 -- 172.21.15.12:0/2618885305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4341e7f20 con 0x7fa434069f30 2023-07-08T12:39:04.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.208+0000 7fa438b4a700 1 -- 172.21.15.12:0/2618885305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4341e8440 con 0x7fa434069f30 2023-07-08T12:39:04.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.209+0000 7fa422ffd700 1 -- 172.21.15.12:0/2618885305 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa42401b430 con 0x7fa434069f30 2023-07-08T12:39:04.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.209+0000 7fa422ffd700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa41406ace0 0x7fa41406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.209+0000 7fa422ffd700 1 -- 172.21.15.12:0/2618885305 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa42408a990 con 0x7fa434069f30 2023-07-08T12:39:04.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.209+0000 7fa431d9b700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa41406ace0 0x7fa41406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.210+0000 7fa431d9b700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa41406ace0 0x7fa41406d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa4340b20b0 tx=0x7fa428008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:04.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.213+0000 7fa422ffd700 1 -- 172.21.15.12:0/2618885305 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa4240557a0 con 0x7fa434069f30 2023-07-08T12:39:04.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.416+0000 7fa438b4a700 1 -- 172.21.15.12:0/2618885305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa43404ea60 con 0x7fa434069f30 2023-07-08T12:39:04.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.417+0000 7fa422ffd700 1 -- 172.21.15.12:0/2618885305 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20203 (secure 0 0 0) 0x7fa424058dc0 con 0x7fa434069f30 2023-07-08T12:39:04.420 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:04.420 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:49.386462+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:39:04.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.419+0000 7fa438b4a700 1 -- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa41406ace0 msgr2=0x7fa41406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.419+0000 7fa438b4a700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa41406ace0 0x7fa41406d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa4340b20b0 tx=0x7fa428008040 comp rx=0 tx=0).stop 2023-07-08T12:39:04.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.419+0000 7fa438b4a700 1 -- 172.21.15.12:0/2618885305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa434069f30 msgr2=0x7fa434151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.419+0000 7fa438b4a700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa434069f30 0x7fa434151280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa424000c00 tx=0x7fa424004750 comp rx=0 tx=0).stop 2023-07-08T12:39:04.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.419+0000 7fa438b4a700 1 -- 172.21.15.12:0/2618885305 shutdown_connections 2023-07-08T12:39:04.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.419+0000 7fa438b4a700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa41406ace0 0x7fa41406d190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.419+0000 7fa438b4a700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa434160790 0x7fa4340cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.419+0000 7fa438b4a700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa434069f30 0x7fa434151280 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.419+0000 7fa438b4a700 1 --2- 172.21.15.12:0/2618885305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa434159c30 0x7fa4340b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.419+0000 7fa438b4a700 1 -- 172.21.15.12:0/2618885305 >> 172.21.15.12:0/2618885305 conn(0x7fa4340cc290 msgr2=0x7fa43415ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:04.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.419+0000 7fa438b4a700 1 -- 172.21.15.12:0/2618885305 shutdown_connections 2023-07-08T12:39:04.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.419+0000 7fa438b4a700 1 -- 172.21.15.12:0/2618885305 wait complete. 2023-07-08T12:39:04.434 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-08T12:38:05.660194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-08T12:38:06.920177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-08T12:38:10.592749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-08T12:39:04.435 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:39:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.562+0000 7f714808b700 1 Processor -- start 2023-07-08T12:39:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.563+0000 7f714808b700 1 -- start start 2023-07-08T12:39:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.563+0000 7f714808b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f71400ce560 0x7f71400ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.563+0000 7f714808b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f714015a6d0 0x7f714015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.563+0000 7f714808b700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f714006f330 0x7f714006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.563+0000 7f714808b700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714015b730 con 0x7f71400ce560 2023-07-08T12:39:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.563+0000 7f714808b700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7140158130 con 0x7f714006f330 2023-07-08T12:39:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.563+0000 7f714808b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71400cee70 con 0x7f714015a6d0 2023-07-08T12:39:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.563+0000 7f7145e27700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f71400ce560 0x7f71400ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.564+0000 7f7145e27700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f71400ce560 0x7f71400ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50780/0 (socket says 172.21.15.12:50780) 2023-07-08T12:39:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.564+0000 7f7145e27700 1 -- 172.21.15.12:0/974487944 learned_addr learned my addr 172.21.15.12:0/974487944 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.564+0000 7f7145626700 1 --2- 172.21.15.12:0/974487944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f714015a6d0 0x7f714015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.564+0000 7f7146628700 1 --2- 172.21.15.12:0/974487944 >> v2:172.21.15.160:3300/0 conn(0x7f714006f330 0x7f714006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.564+0000 7f7145e27700 1 -- 172.21.15.12:0/974487944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f714015a6d0 msgr2=0x7f714015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.564+0000 7f7145e27700 1 --2- 172.21.15.12:0/974487944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f714015a6d0 0x7f714015ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.564+0000 7f7145e27700 1 -- 172.21.15.12:0/974487944 >> v2:172.21.15.160:3300/0 conn(0x7f714006f330 msgr2=0x7f714006f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.564+0000 7f7145e27700 1 --2- 172.21.15.12:0/974487944 >> v2:172.21.15.160:3300/0 conn(0x7f714006f330 0x7f714006f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.564+0000 7f7145e27700 1 -- 172.21.15.12:0/974487944 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714015b450 con 0x7f71400ce560 2023-07-08T12:39:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.564+0000 7f7145e27700 1 --2- 172.21.15.12:0/974487944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71400ce560 0x7f71400ce930 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f71400b0340 tx=0x7f713c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc28f24597d34c8e server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.564+0000 7f7144e25700 1 -- 172.21.15.12:0/974487944 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c011830 con 0x7f71400ce560 2023-07-08T12:39:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.564+0000 7f7144e25700 1 -- 172.21.15.12:0/974487944 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c011990 con 0x7f71400ce560 2023-07-08T12:39:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.564+0000 7f7144e25700 1 -- 172.21.15.12:0/974487944 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c011d20 con 0x7f71400ce560 2023-07-08T12:39:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.565+0000 7f714808b700 1 -- 172.21.15.12:0/974487944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71400ce560 msgr2=0x7f71400ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.565+0000 7f714808b700 1 --2- 172.21.15.12:0/974487944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71400ce560 0x7f71400ce930 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f71400b0340 tx=0x7f713c009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.565+0000 7f714808b700 1 -- 172.21.15.12:0/974487944 shutdown_connections 2023-07-08T12:39:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.565+0000 7f714808b700 1 --2- 172.21.15.12:0/974487944 >> v2:172.21.15.160:3300/0 conn(0x7f714006f330 0x7f714006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.565+0000 7f714808b700 1 --2- 172.21.15.12:0/974487944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71400ce560 0x7f71400ce930 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.565+0000 7f714808b700 1 --2- 172.21.15.12:0/974487944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f714015a6d0 0x7f714015ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.565+0000 7f714808b700 1 -- 172.21.15.12:0/974487944 >> 172.21.15.12:0/974487944 conn(0x7f7140099c60 msgr2=0x7f7140089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.565+0000 7f714808b700 1 -- 172.21.15.12:0/974487944 shutdown_connections 2023-07-08T12:39:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.565+0000 7f714808b700 1 -- 172.21.15.12:0/974487944 wait complete. 2023-07-08T12:39:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.567+0000 7f714808b700 1 Processor -- start 2023-07-08T12:39:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.567+0000 7f714808b700 1 -- start start 2023-07-08T12:39:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f714808b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f714006f330 0x7f71400b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f714808b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71400ce560 0x7f71400cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f714808b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f714015a6d0 0x7f71401567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f714808b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71400b3fc0 con 0x7f714015a6d0 2023-07-08T12:39:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f714808b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714006c010 con 0x7f71400ce560 2023-07-08T12:39:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f714808b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71400ccdc0 con 0x7f714006f330 2023-07-08T12:39:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f7146628700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f714015a6d0 0x7f71401567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f7146628700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f714015a6d0 0x7f71401567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50790/0 (socket says 172.21.15.12:50790) 2023-07-08T12:39:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f7146628700 1 -- 172.21.15.12:0/3461868718 learned_addr learned my addr 172.21.15.12:0/3461868718 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:04.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f7145e27700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f714006f330 0x7f71400b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f7145626700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71400ce560 0x7f71400cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f7146628700 1 -- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f714006f330 msgr2=0x7f71400b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f7146628700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f714006f330 0x7f71400b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f7146628700 1 -- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71400ce560 msgr2=0x7f71400cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f7146628700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71400ce560 0x7f71400cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f7146628700 1 -- 172.21.15.12:0/3461868718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c009550 con 0x7f714015a6d0 2023-07-08T12:39:04.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f7146628700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f714015a6d0 0x7f71401567f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f713800eed0 tx=0x7f713800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:04.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.568+0000 7f71367fc700 1 -- 172.21.15.12:0/3461868718 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138010040 con 0x7f714015a6d0 2023-07-08T12:39:04.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.569+0000 7f71367fc700 1 -- 172.21.15.12:0/3461868718 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713800b9f0 con 0x7f714015a6d0 2023-07-08T12:39:04.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.569+0000 7f714808b700 1 -- 172.21.15.12:0/3461868718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f714006ba50 con 0x7f714015a6d0 2023-07-08T12:39:04.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.569+0000 7f71367fc700 1 -- 172.21.15.12:0/3461868718 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713800bda0 con 0x7f714015a6d0 2023-07-08T12:39:04.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.569+0000 7f714808b700 1 -- 172.21.15.12:0/3461868718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714006b170 con 0x7f714015a6d0 2023-07-08T12:39:04.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.570+0000 7f714808b700 1 -- 172.21.15.12:0/3461868718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71400b14d0 con 0x7f714015a6d0 2023-07-08T12:39:04.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.570+0000 7f71367fc700 1 -- 172.21.15.12:0/3461868718 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f713801d400 con 0x7f714015a6d0 2023-07-08T12:39:04.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.570+0000 7f71367fc700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f712406ace0 0x7f712406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.571+0000 7f71367fc700 1 -- 172.21.15.12:0/3461868718 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7138089d70 con 0x7f714015a6d0 2023-07-08T12:39:04.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.571+0000 7f7145e27700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f712406ace0 0x7f712406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.571+0000 7f7145e27700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f712406ace0 0x7f712406d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f713c009840 tx=0x7f713c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:04.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.575+0000 7f71367fc700 1 -- 172.21.15.12:0/3461868718 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f71400b14d0 con 0x7f714015a6d0 2023-07-08T12:39:04.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.772+0000 7f714808b700 1 -- 172.21.15.12:0/3461868718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71400b14d0 con 0x7f714015a6d0 2023-07-08T12:39:04.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.773+0000 7f71367fc700 1 -- 172.21.15.12:0/3461868718 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7138018020 con 0x7f714015a6d0 2023-07-08T12:39:04.777 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:39:04.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.775+0000 7f714808b700 1 -- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f712406ace0 msgr2=0x7f712406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.775+0000 7f714808b700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f712406ace0 0x7f712406d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f713c009840 tx=0x7f713c008030 comp rx=0 tx=0).stop 2023-07-08T12:39:04.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.775+0000 7f714808b700 1 -- 172.21.15.12:0/3461868718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f714015a6d0 msgr2=0x7f71401567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.775+0000 7f714808b700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f714015a6d0 0x7f71401567f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f713800eed0 tx=0x7f713800ef00 comp rx=0 tx=0).stop 2023-07-08T12:39:04.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.775+0000 7f714808b700 1 -- 172.21.15.12:0/3461868718 shutdown_connections 2023-07-08T12:39:04.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.775+0000 7f714808b700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f712406ace0 0x7f712406d190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.775+0000 7f714808b700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f714015a6d0 0x7f71401567f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.775+0000 7f714808b700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f714006f330 0x7f71400b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.775+0000 7f714808b700 1 --2- 172.21.15.12:0/3461868718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71400ce560 0x7f71400cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.775+0000 7f714808b700 1 -- 172.21.15.12:0/3461868718 >> 172.21.15.12:0/3461868718 conn(0x7f7140099c60 msgr2=0x7f71400729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:04.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.775+0000 7f714808b700 1 -- 172.21.15.12:0/3461868718 shutdown_connections 2023-07-08T12:39:04.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.775+0000 7f714808b700 1 -- 172.21.15.12:0/3461868718 wait complete. 2023-07-08T12:39:04.789 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:39:04.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.915+0000 7f72b9608700 1 Processor -- start 2023-07-08T12:39:04.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.915+0000 7f72b9608700 1 -- start start 2023-07-08T12:39:04.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.915+0000 7f72b9608700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f72b40cbe90 0x7f72b40cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.915+0000 7f72b9608700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f72b40b2100 0x7f72b40b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.915+0000 7f72b9608700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b4069f30 con 0x7f72b415a210 2023-07-08T12:39:04.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.915+0000 7f72b9608700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b40cb760 con 0x7f72b40b2100 2023-07-08T12:39:04.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.915+0000 7f72b9608700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b40cc840 con 0x7f72b40cbe90 2023-07-08T12:39:04.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b37fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f72b40b2100 0x7f72b40b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b2ffd700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f72b415a210 0x7f72b40c9640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38190/0 (socket says 172.21.15.12:38190) 2023-07-08T12:39:04.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b37fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f72b40b2100 0x7f72b40b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50798/0 (socket says 172.21.15.12:50798) 2023-07-08T12:39:04.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b2ffd700 1 -- 172.21.15.12:0/1598807057 learned_addr learned my addr 172.21.15.12:0/1598807057 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:04.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b27fc700 1 --2- 172.21.15.12:0/1598807057 >> v2:172.21.15.160:3300/0 conn(0x7f72b40cbe90 0x7f72b40cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b37fe700 1 -- 172.21.15.12:0/1598807057 >> v2:172.21.15.160:3300/0 conn(0x7f72b40cbe90 msgr2=0x7f72b40cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:04.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b37fe700 1 --2- 172.21.15.12:0/1598807057 >> v2:172.21.15.160:3300/0 conn(0x7f72b40cbe90 0x7f72b40cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b37fe700 1 -- 172.21.15.12:0/1598807057 >> v1:172.21.15.12:6789/0 conn(0x7f72b415a210 legacy=0x7f72b40c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b37fe700 1 -- 172.21.15.12:0/1598807057 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b406a1d0 con 0x7f72b40b2100 2023-07-08T12:39:04.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b1ffb700 1 -- 172.21.15.12:0/1598807057 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282642167 0 0) 0x7f72b4069f30 con 0x7f72b415a210 2023-07-08T12:39:04.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b27fc700 1 --2- 172.21.15.12:0/1598807057 >> v2:172.21.15.160:3300/0 conn(0x7f72b40cbe90 0x7f72b40cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:04.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b37fe700 1 --2- 172.21.15.12:0/1598807057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72b40b2100 0x7f72b40b2560 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f72b406e4d0 tx=0x7f72ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f133368eedd5188 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:04.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b1ffb700 1 -- 172.21.15.12:0/1598807057 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00e070 con 0x7f72b40b2100 2023-07-08T12:39:04.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b1ffb700 1 -- 172.21.15.12:0/1598807057 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac00bbd0 con 0x7f72b40b2100 2023-07-08T12:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.916+0000 7f72b1ffb700 1 -- 172.21.15.12:0/1598807057 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac012430 con 0x7f72b40b2100 2023-07-08T12:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.917+0000 7f72b9608700 1 -- 172.21.15.12:0/1598807057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72b40b2100 msgr2=0x7f72b40b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.917+0000 7f72b9608700 1 --2- 172.21.15.12:0/1598807057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72b40b2100 0x7f72b40b2560 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f72b406e4d0 tx=0x7f72ac00b330 comp rx=0 tx=0).stop 2023-07-08T12:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.917+0000 7f72b9608700 1 -- 172.21.15.12:0/1598807057 shutdown_connections 2023-07-08T12:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.917+0000 7f72b9608700 1 --2- 172.21.15.12:0/1598807057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72b40b2100 0x7f72b40b2560 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.917+0000 7f72b9608700 1 --2- 172.21.15.12:0/1598807057 >> v2:172.21.15.160:3300/0 conn(0x7f72b40cbe90 0x7f72b40cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.917+0000 7f72b9608700 1 -- 172.21.15.12:0/1598807057 >> 172.21.15.12:0/1598807057 conn(0x7f72b40cf1d0 msgr2=0x7f72b40cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:04.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.917+0000 7f72b9608700 1 -- 172.21.15.12:0/1598807057 shutdown_connections 2023-07-08T12:39:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.917+0000 7f72b9608700 1 -- 172.21.15.12:0/1598807057 wait complete. 2023-07-08T12:39:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.918+0000 7f72b9608700 1 Processor -- start 2023-07-08T12:39:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.918+0000 7f72b9608700 1 -- start start 2023-07-08T12:39:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b9608700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72b40b2100 0x7f72b41560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b9608700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f72b40cbe90 0x7f72b4152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b9608700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72b415a210 0x7f72b4153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b9608700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b4150e20 con 0x7f72b40b2100 2023-07-08T12:39:04.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b9608700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b41510c0 con 0x7f72b415a210 2023-07-08T12:39:04.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b9608700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b41513a0 con 0x7f72b40cbe90 2023-07-08T12:39:04.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72b40b2100 0x7f72b41560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72b40b2100 0x7f72b41560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50800/0 (socket says 172.21.15.12:50800) 2023-07-08T12:39:04.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b2ffd700 1 -- 172.21.15.12:0/1256918300 learned_addr learned my addr 172.21.15.12:0/1256918300 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:04.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b27fc700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f72b40cbe90 0x7f72b4152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b37fe700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72b415a210 0x7f72b4153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b2ffd700 1 -- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f72b40cbe90 msgr2=0x7f72b4152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b2ffd700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f72b40cbe90 0x7f72b4152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b2ffd700 1 -- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72b415a210 msgr2=0x7f72b4153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:04.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b2ffd700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72b415a210 0x7f72b4153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:04.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.919+0000 7f72b2ffd700 1 -- 172.21.15.12:0/1256918300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac009a70 con 0x7f72b40b2100 2023-07-08T12:39:04.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.920+0000 7f72b37fe700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72b415a210 0x7f72b4153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:04.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.920+0000 7f72b2ffd700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72b40b2100 0x7f72b41560f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f72a400c550 tx=0x7f72a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:04.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.920+0000 7f729b7fe700 1 -- 172.21.15.12:0/1256918300 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a4004c80 con 0x7f72b40b2100 2023-07-08T12:39:04.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.920+0000 7f729b7fe700 1 -- 172.21.15.12:0/1256918300 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a4004e00 con 0x7f72b40b2100 2023-07-08T12:39:04.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.920+0000 7f72b9608700 1 -- 172.21.15.12:0/1256918300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b4151680 con 0x7f72b40b2100 2023-07-08T12:39:04.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.920+0000 7f729b7fe700 1 -- 172.21.15.12:0/1256918300 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a4007310 con 0x7f72b40b2100 2023-07-08T12:39:04.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.920+0000 7f72b9608700 1 -- 172.21.15.12:0/1256918300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b4151c40 con 0x7f72b40b2100 2023-07-08T12:39:04.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.921+0000 7f72b9608700 1 -- 172.21.15.12:0/1256918300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b4151e60 con 0x7f72b40b2100 2023-07-08T12:39:04.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.921+0000 7f729b7fe700 1 -- 172.21.15.12:0/1256918300 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f72a4007470 con 0x7f72b40b2100 2023-07-08T12:39:04.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.922+0000 7f729b7fe700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f72a006ace0 0x7f72a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:04.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.922+0000 7f729b7fe700 1 -- 172.21.15.12:0/1256918300 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f72a40894c0 con 0x7f72b40b2100 2023-07-08T12:39:04.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.922+0000 7f72b27fc700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f72a006ace0 0x7f72a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:04.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.923+0000 7f72b27fc700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f72a006ace0 0x7f72a006d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f72a8006fd0 tx=0x7f72a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:04.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:04.926+0000 7f729b7fe700 1 -- 172.21.15.12:0/1256918300 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f72a4054240 con 0x7f72b40b2100 2023-07-08T12:39:05.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.122+0000 7f72b9608700 1 -- 172.21.15.12:0/1256918300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f72b404ea60 con 0x7f72b40b2100 2023-07-08T12:39:05.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.123+0000 7f729b7fe700 1 -- 172.21.15.12:0/1256918300 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f72a4057860 con 0x7f72b40b2100 2023-07-08T12:39:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:39:05.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.125+0000 7f72b9608700 1 -- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f72a006ace0 msgr2=0x7f72a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.125+0000 7f72b9608700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f72a006ace0 0x7f72a006d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f72a8006fd0 tx=0x7f72a8008040 comp rx=0 tx=0).stop 2023-07-08T12:39:05.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.125+0000 7f72b9608700 1 -- 172.21.15.12:0/1256918300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72b40b2100 msgr2=0x7f72b41560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.125+0000 7f72b9608700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72b40b2100 0x7f72b41560f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f72a400c550 tx=0x7f72a400c910 comp rx=0 tx=0).stop 2023-07-08T12:39:05.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.125+0000 7f72b9608700 1 -- 172.21.15.12:0/1256918300 shutdown_connections 2023-07-08T12:39:05.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.125+0000 7f72b9608700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f72a006ace0 0x7f72a006d190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.125+0000 7f72b9608700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72b415a210 0x7f72b4153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.125+0000 7f72b9608700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72b40b2100 0x7f72b41560f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.125+0000 7f72b9608700 1 --2- 172.21.15.12:0/1256918300 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f72b40cbe90 0x7f72b4152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.125+0000 7f72b9608700 1 -- 172.21.15.12:0/1256918300 >> 172.21.15.12:0/1256918300 conn(0x7f72b40cf1d0 msgr2=0x7f72b40b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:05.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.125+0000 7f72b9608700 1 -- 172.21.15.12:0/1256918300 shutdown_connections 2023-07-08T12:39:05.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.125+0000 7f72b9608700 1 -- 172.21.15.12:0/1256918300 wait complete. 2023-07-08T12:39:05.139 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-08T12:39:05.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.265+0000 7f22f0ca7700 1 Processor -- start 2023-07-08T12:39:05.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.265+0000 7f22f0ca7700 1 -- start start 2023-07-08T12:39:05.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.266+0000 7f22f0ca7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ec15fad0 0x7f22ec0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.266+0000 7f22f0ca7700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f22ec06b750 0x7f22ec0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.266+0000 7f22f0ca7700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ec15fea0 con 0x7f22ec0ce130 2023-07-08T12:39:05.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.266+0000 7f22f0ca7700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ec06e750 con 0x7f22ec06b750 2023-07-08T12:39:05.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.266+0000 7f22f0ca7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec06ea30 con 0x7f22ec15fad0 2023-07-08T12:39:05.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.266+0000 7f22ea59c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ec15fad0 0x7f22ec0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.266+0000 7f22ead9d700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f22ec06b750 0x7f22ec0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.266+0000 7f22ea59c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ec15fad0 0x7f22ec0cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:60426/0 (socket says 172.21.15.12:60426) 2023-07-08T12:39:05.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.266+0000 7f22e9d9b700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f22ec0ce130 0x7f22ec06b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:47428/0 (socket says 172.21.15.12:47428) 2023-07-08T12:39:05.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.267+0000 7f22ea59c700 1 -- 172.21.15.12:0/776022869 learned_addr learned my addr 172.21.15.12:0/776022869 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:05.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.267+0000 7f22ea59c700 1 -- 172.21.15.12:0/776022869 >> v2:172.21.15.160:3300/0 conn(0x7f22ec06b750 msgr2=0x7f22ec0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:05.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.267+0000 7f22ea59c700 1 --2- 172.21.15.12:0/776022869 >> v2:172.21.15.160:3300/0 conn(0x7f22ec06b750 0x7f22ec0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.267+0000 7f22ea59c700 1 -- 172.21.15.12:0/776022869 >> v1:172.21.15.160:6789/0 conn(0x7f22ec0ce130 legacy=0x7f22ec06b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.267+0000 7f22e959a700 1 -- 172.21.15.12:0/776022869 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160504291 0 0) 0x7f22ec15fea0 con 0x7f22ec0ce130 2023-07-08T12:39:05.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.267+0000 7f22ea59c700 1 -- 172.21.15.12:0/776022869 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec06ed10 con 0x7f22ec15fad0 2023-07-08T12:39:05.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.267+0000 7f22ead9d700 1 --2- 172.21.15.12:0/776022869 >> v2:172.21.15.160:3300/0 conn(0x7f22ec06b750 0x7f22ec0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:39:05.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.267+0000 7f22ea59c700 1 --2- 172.21.15.12:0/776022869 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ec15fad0 0x7f22ec0cdbf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0098f0 tx=0x7f22dc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8590d4be38bd0d78 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:05.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.267+0000 7f22e959a700 1 -- 172.21.15.12:0/776022869 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc011830 con 0x7f22ec15fad0 2023-07-08T12:39:05.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.267+0000 7f22e959a700 1 -- 172.21.15.12:0/776022869 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc011990 con 0x7f22ec15fad0 2023-07-08T12:39:05.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.267+0000 7f22e959a700 1 -- 172.21.15.12:0/776022869 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc011d20 con 0x7f22ec15fad0 2023-07-08T12:39:05.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.268+0000 7f22f0ca7700 1 -- 172.21.15.12:0/776022869 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ec15fad0 msgr2=0x7f22ec0cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.268+0000 7f22f0ca7700 1 --2- 172.21.15.12:0/776022869 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ec15fad0 0x7f22ec0cdbf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0098f0 tx=0x7f22dc009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:05.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.268+0000 7f22f0ca7700 1 -- 172.21.15.12:0/776022869 shutdown_connections 2023-07-08T12:39:05.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.268+0000 7f22f0ca7700 1 --2- 172.21.15.12:0/776022869 >> v2:172.21.15.160:3300/0 conn(0x7f22ec06b750 0x7f22ec0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.268+0000 7f22f0ca7700 1 --2- 172.21.15.12:0/776022869 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ec15fad0 0x7f22ec0cdbf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.268+0000 7f22f0ca7700 1 -- 172.21.15.12:0/776022869 >> 172.21.15.12:0/776022869 conn(0x7f22ec099c60 msgr2=0x7f22ec089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:05.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.268+0000 7f22f0ca7700 1 -- 172.21.15.12:0/776022869 shutdown_connections 2023-07-08T12:39:05.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.268+0000 7f22f0ca7700 1 -- 172.21.15.12:0/776022869 wait complete. 2023-07-08T12:39:05.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.269+0000 7f22f0ca7700 1 Processor -- start 2023-07-08T12:39:05.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.269+0000 7f22f0ca7700 1 -- start start 2023-07-08T12:39:05.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.270+0000 7f22f0ca7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ec06b750 0x7f22ec0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.270+0000 7f22f0ca7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec0ce130 0x7f22ec0c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.270+0000 7f22f0ca7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ec15fad0 0x7f22ec1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.270+0000 7f22f0ca7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0c7fa0 con 0x7f22ec15fad0 2023-07-08T12:39:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.270+0000 7f22f0ca7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0c81b0 con 0x7f22ec0ce130 2023-07-08T12:39:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.270+0000 7f22f0ca7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0c8490 con 0x7f22ec06b750 2023-07-08T12:39:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.270+0000 7f22ead9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ec15fad0 0x7f22ec1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.270+0000 7f22ead9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ec15fad0 0x7f22ec1579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50818/0 (socket says 172.21.15.12:50818) 2023-07-08T12:39:05.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.270+0000 7f22ead9d700 1 -- 172.21.15.12:0/966901130 learned_addr learned my addr 172.21.15.12:0/966901130 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.270+0000 7f22ea59c700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ec06b750 0x7f22ec0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.270+0000 7f22e9d9b700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec0ce130 0x7f22ec0c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.271+0000 7f22ead9d700 1 -- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ec06b750 msgr2=0x7f22ec0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.271+0000 7f22ead9d700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ec06b750 0x7f22ec0ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.271+0000 7f22ead9d700 1 -- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec0ce130 msgr2=0x7f22ec0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.271+0000 7f22ead9d700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec0ce130 0x7f22ec0c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.271+0000 7f22ead9d700 1 -- 172.21.15.12:0/966901130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e4009a70 con 0x7f22ec15fad0 2023-07-08T12:39:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.271+0000 7f22ead9d700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ec15fad0 0x7f22ec1579f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f22e400efc0 tx=0x7f22e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.271+0000 7f22e2ffd700 1 -- 172.21.15.12:0/966901130 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e400cc90 con 0x7f22ec15fad0 2023-07-08T12:39:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.271+0000 7f22e2ffd700 1 -- 172.21.15.12:0/966901130 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e400f040 con 0x7f22ec15fad0 2023-07-08T12:39:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.271+0000 7f22f0ca7700 1 -- 172.21.15.12:0/966901130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22dc009550 con 0x7f22ec15fad0 2023-07-08T12:39:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.271+0000 7f22e2ffd700 1 -- 172.21.15.12:0/966901130 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e40146c0 con 0x7f22ec15fad0 2023-07-08T12:39:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.271+0000 7f22f0ca7700 1 -- 172.21.15.12:0/966901130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec0c8a50 con 0x7f22ec15fad0 2023-07-08T12:39:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.272+0000 7f22f0ca7700 1 -- 172.21.15.12:0/966901130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec0b0b20 con 0x7f22ec15fad0 2023-07-08T12:39:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.272+0000 7f22e2ffd700 1 -- 172.21.15.12:0/966901130 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f22e4014820 con 0x7f22ec15fad0 2023-07-08T12:39:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.273+0000 7f22e2ffd700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f22cc06ace0 0x7f22cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.273+0000 7f22e2ffd700 1 -- 172.21.15.12:0/966901130 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f22e408a450 con 0x7f22ec15fad0 2023-07-08T12:39:05.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.273+0000 7f22ea59c700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f22cc06ace0 0x7f22cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.273+0000 7f22ea59c700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f22cc06ace0 0x7f22cc06d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f22dc014040 tx=0x7f22dc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:05.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.277+0000 7f22e2ffd700 1 -- 172.21.15.12:0/966901130 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f22ec0b0b20 con 0x7f22ec15fad0 2023-07-08T12:39:05.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.475+0000 7f22f0ca7700 1 -- 172.21.15.12:0/966901130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22ec0b0b20 con 0x7f22ec15fad0 2023-07-08T12:39:05.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.476+0000 7f22e2ffd700 1 -- 172.21.15.12:0/966901130 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f22e4018020 con 0x7f22ec15fad0 2023-07-08T12:39:05.478 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:05.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.478+0000 7f22f0ca7700 1 -- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f22cc06ace0 msgr2=0x7f22cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.478+0000 7f22f0ca7700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f22cc06ace0 0x7f22cc06d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f22dc014040 tx=0x7f22dc008030 comp rx=0 tx=0).stop 2023-07-08T12:39:05.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.478+0000 7f22f0ca7700 1 -- 172.21.15.12:0/966901130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ec15fad0 msgr2=0x7f22ec1579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.478+0000 7f22f0ca7700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ec15fad0 0x7f22ec1579f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f22e400efc0 tx=0x7f22e400c3a0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.478+0000 7f22f0ca7700 1 -- 172.21.15.12:0/966901130 shutdown_connections 2023-07-08T12:39:05.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.478+0000 7f22f0ca7700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f22cc06ace0 0x7f22cc06d190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.478+0000 7f22f0ca7700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ec15fad0 0x7f22ec1579f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.478+0000 7f22f0ca7700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ec06b750 0x7f22ec0ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.478+0000 7f22f0ca7700 1 --2- 172.21.15.12:0/966901130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ec0ce130 0x7f22ec0c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.478+0000 7f22f0ca7700 1 -- 172.21.15.12:0/966901130 >> 172.21.15.12:0/966901130 conn(0x7f22ec099c60 msgr2=0x7f22ec0ad550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:05.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.478+0000 7f22f0ca7700 1 -- 172.21.15.12:0/966901130 shutdown_connections 2023-07-08T12:39:05.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.478+0000 7f22f0ca7700 1 -- 172.21.15.12:0/966901130 wait complete. 2023-07-08T12:39:05.492 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-08T12:39:05.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.619+0000 7f5784a64700 1 Processor -- start 2023-07-08T12:39:05.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.619+0000 7f5784a64700 1 -- start start 2023-07-08T12:39:05.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f5784a64700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5780156a10 0x7f578006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f5784a64700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57800cc240 0x7f57800cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f5784a64700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5780160410 con 0x7f578015d390 2023-07-08T12:39:05.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f5784a64700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57800ccc00 con 0x7f5780156a10 2023-07-08T12:39:05.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f5784a64700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5780156de0 con 0x7f57800cc240 2023-07-08T12:39:05.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f577e59c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5780156a10 0x7f578006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f577e59c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5780156a10 0x7f578006e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50822/0 (socket says 172.21.15.12:50822) 2023-07-08T12:39:05.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f577e59c700 1 -- 172.21.15.12:0/3266645606 learned_addr learned my addr 172.21.15.12:0/3266645606 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:05.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f577ed9d700 1 --2- 172.21.15.12:0/3266645606 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57800cc240 0x7f57800cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f577e59c700 1 -- 172.21.15.12:0/3266645606 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57800cc240 msgr2=0x7f57800cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:05.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f577e59c700 1 --2- 172.21.15.12:0/3266645606 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57800cc240 0x7f57800cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f577e59c700 1 -- 172.21.15.12:0/3266645606 >> v1:172.21.15.160:6789/0 conn(0x7f578015d390 legacy=0x7f57801608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f577e59c700 1 -- 172.21.15.12:0/3266645606 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578006ec70 con 0x7f5780156a10 2023-07-08T12:39:05.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f577ed9d700 1 --2- 172.21.15.12:0/3266645606 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57800cc240 0x7f57800cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:05.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f577e59c700 1 --2- 172.21.15.12:0/3266645606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5780156a10 0x7f578006e0c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5770006fd0 tx=0x7f5770009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a1a89c64033c04 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:05.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.620+0000 7f577d59a700 1 -- 172.21.15.12:0/3266645606 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770005e30 con 0x7f5780156a10 2023-07-08T12:39:05.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.621+0000 7f577d59a700 1 -- 172.21.15.12:0/3266645606 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5770004030 con 0x7f5780156a10 2023-07-08T12:39:05.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.621+0000 7f577d59a700 1 -- 172.21.15.12:0/3266645606 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57700043e0 con 0x7f5780156a10 2023-07-08T12:39:05.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.621+0000 7f5784a64700 1 -- 172.21.15.12:0/3266645606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5780156a10 msgr2=0x7f578006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.621+0000 7f5784a64700 1 --2- 172.21.15.12:0/3266645606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5780156a10 0x7f578006e0c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5770006fd0 tx=0x7f5770009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.621+0000 7f5784a64700 1 -- 172.21.15.12:0/3266645606 shutdown_connections 2023-07-08T12:39:05.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.621+0000 7f5784a64700 1 --2- 172.21.15.12:0/3266645606 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57800cc240 0x7f57800cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.621+0000 7f5784a64700 1 --2- 172.21.15.12:0/3266645606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5780156a10 0x7f578006e0c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.621+0000 7f5784a64700 1 -- 172.21.15.12:0/3266645606 >> 172.21.15.12:0/3266645606 conn(0x7f5780098b50 msgr2=0x7f578006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:05.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.621+0000 7f5784a64700 1 -- 172.21.15.12:0/3266645606 shutdown_connections 2023-07-08T12:39:05.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.621+0000 7f5784a64700 1 -- 172.21.15.12:0/3266645606 wait complete. 2023-07-08T12:39:05.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.622+0000 7f5784a64700 1 Processor -- start 2023-07-08T12:39:05.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.623+0000 7f5784a64700 1 -- start start 2023-07-08T12:39:05.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.623+0000 7f5784a64700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57800cc240 0x7f57800b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.623+0000 7f5784a64700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5780156a10 0x7f5780155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.623+0000 7f5784a64700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578015d390 0x7f578015e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.623+0000 7f5784a64700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57801f6810 con 0x7f578015d390 2023-07-08T12:39:05.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.623+0000 7f5784a64700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57800b2bf0 con 0x7f57800cc240 2023-07-08T12:39:05.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.623+0000 7f5784a64700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57801560c0 con 0x7f5780156a10 2023-07-08T12:39:05.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.623+0000 7f577ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578015d390 0x7f578015e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.623+0000 7f577ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578015d390 0x7f578015e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50832/0 (socket says 172.21.15.12:50832) 2023-07-08T12:39:05.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.623+0000 7f577ed9d700 1 -- 172.21.15.12:0/659193790 learned_addr learned my addr 172.21.15.12:0/659193790 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:05.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.624+0000 7f577e59c700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57800cc240 0x7f57800b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.624+0000 7f577dd9b700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5780156a10 0x7f5780155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.624+0000 7f577ed9d700 1 -- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5780156a10 msgr2=0x7f5780155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.624+0000 7f577ed9d700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5780156a10 0x7f5780155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.624+0000 7f577ed9d700 1 -- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57800cc240 msgr2=0x7f57800b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.624+0000 7f577ed9d700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57800cc240 0x7f57800b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.624+0000 7f577ed9d700 1 -- 172.21.15.12:0/659193790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5770009500 con 0x7f578015d390 2023-07-08T12:39:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.624+0000 7f577ed9d700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578015d390 0x7f578015e960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5778007ef0 tx=0x7f577800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.624+0000 7f576effd700 1 -- 172.21.15.12:0/659193790 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577800dd80 con 0x7f578015d390 2023-07-08T12:39:05.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.624+0000 7f576effd700 1 -- 172.21.15.12:0/659193790 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577800f040 con 0x7f578015d390 2023-07-08T12:39:05.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.624+0000 7f576effd700 1 -- 172.21.15.12:0/659193790 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778003c60 con 0x7f578015d390 2023-07-08T12:39:05.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.624+0000 7f5784a64700 1 -- 172.21.15.12:0/659193790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57801e5f00 con 0x7f578015d390 2023-07-08T12:39:05.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.624+0000 7f5784a64700 1 -- 172.21.15.12:0/659193790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57801e64c0 con 0x7f578015d390 2023-07-08T12:39:05.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.625+0000 7f5784a64700 1 -- 172.21.15.12:0/659193790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57800b22c0 con 0x7f578015d390 2023-07-08T12:39:05.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.626+0000 7f576effd700 1 -- 172.21.15.12:0/659193790 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5778004030 con 0x7f578015d390 2023-07-08T12:39:05.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.626+0000 7f576effd700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f576006ace0 0x7f576006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.626+0000 7f576effd700 1 -- 172.21.15.12:0/659193790 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5778089ee0 con 0x7f578015d390 2023-07-08T12:39:05.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.627+0000 7f577e59c700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f576006ace0 0x7f576006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.627+0000 7f577e59c700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f576006ace0 0x7f576006d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5770009130 tx=0x7f5770008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:05.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.630+0000 7f576effd700 1 -- 172.21.15.12:0/659193790 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5778054cd0 con 0x7f578015d390 2023-07-08T12:39:05.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.827+0000 7f5784a64700 1 -- 172.21.15.12:0/659193790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57800b2d60 con 0x7f578015d390 2023-07-08T12:39:05.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.827+0000 7f576effd700 1 -- 172.21.15.12:0/659193790 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57800b2d60 con 0x7f578015d390 2023-07-08T12:39:05.830 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-07-08T12:39:05.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.829+0000 7f5784a64700 1 -- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f576006ace0 msgr2=0x7f576006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.829+0000 7f5784a64700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f576006ace0 0x7f576006d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5770009130 tx=0x7f5770008070 comp rx=0 tx=0).stop 2023-07-08T12:39:05.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.829+0000 7f5784a64700 1 -- 172.21.15.12:0/659193790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578015d390 msgr2=0x7f578015e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.829+0000 7f5784a64700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578015d390 0x7f578015e960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5778007ef0 tx=0x7f577800d3b0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.829+0000 7f5784a64700 1 -- 172.21.15.12:0/659193790 shutdown_connections 2023-07-08T12:39:05.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.829+0000 7f5784a64700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f576006ace0 0x7f576006d190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.829+0000 7f5784a64700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578015d390 0x7f578015e960 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.829+0000 7f5784a64700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57800cc240 0x7f57800b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.829+0000 7f5784a64700 1 --2- 172.21.15.12:0/659193790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5780156a10 0x7f5780155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.829+0000 7f5784a64700 1 -- 172.21.15.12:0/659193790 >> 172.21.15.12:0/659193790 conn(0x7f5780098b50 msgr2=0x7f578006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:05.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.829+0000 7f5784a64700 1 -- 172.21.15.12:0/659193790 shutdown_connections 2023-07-08T12:39:05.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.829+0000 7f5784a64700 1 -- 172.21.15.12:0/659193790 wait complete. 2023-07-08T12:39:05.844 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-08T12:39:05.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.970+0000 7f65a7910700 1 Processor -- start 2023-07-08T12:39:05.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.970+0000 7f65a7910700 1 -- start start 2023-07-08T12:39:05.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.971+0000 7f65a7910700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f65a00aeec0 0x7f65a006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.971+0000 7f65a7910700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65a00cdb80 0x7f65a00cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.971+0000 7f65a7910700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a015d770 con 0x7f65a006ebe0 2023-07-08T12:39:05.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.971+0000 7f65a7910700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a015a600 con 0x7f65a00aeec0 2023-07-08T12:39:05.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.971+0000 7f65a7910700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a015cb60 con 0x7f65a00cdb80 2023-07-08T12:39:05.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.971+0000 7f65a56ac700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f65a00aeec0 0x7f65a006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.971+0000 7f65a5ead700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65a00cdb80 0x7f65a00cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.971+0000 7f65a4eab700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f65a006ebe0 0x7f65a00cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:47454/0 (socket says 172.21.15.12:47454) 2023-07-08T12:39:05.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.971+0000 7f65a4eab700 1 -- 172.21.15.12:0/37003666 learned_addr learned my addr 172.21.15.12:0/37003666 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:05.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.971+0000 7f65a56ac700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f65a00aeec0 0x7f65a006cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49580/0 (socket says 172.21.15.12:49580) 2023-07-08T12:39:05.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.972+0000 7f65a5ead700 1 -- 172.21.15.12:0/37003666 >> v2:172.21.15.160:3300/0 conn(0x7f65a00aeec0 msgr2=0x7f65a006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:05.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.972+0000 7f65a5ead700 1 --2- 172.21.15.12:0/37003666 >> v2:172.21.15.160:3300/0 conn(0x7f65a00aeec0 0x7f65a006cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.972+0000 7f65a5ead700 1 -- 172.21.15.12:0/37003666 >> v1:172.21.15.160:6789/0 conn(0x7f65a006ebe0 legacy=0x7f65a00cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.972+0000 7f65a5ead700 1 -- 172.21.15.12:0/37003666 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a015ce40 con 0x7f65a00cdb80 2023-07-08T12:39:05.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.972+0000 7f65a56ac700 1 --2- 172.21.15.12:0/37003666 >> v2:172.21.15.160:3300/0 conn(0x7f65a00aeec0 0x7f65a006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:39:05.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.972+0000 7f65a5ead700 1 --2- 172.21.15.12:0/37003666 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65a00cdb80 0x7f65a00cbe10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6598009d90 tx=0x7f659800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75d2fec5075e76c server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:05.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.972+0000 7f6597fff700 1 -- 172.21.15.12:0/37003666 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659800e070 con 0x7f65a00cdb80 2023-07-08T12:39:05.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.972+0000 7f6597fff700 1 -- 172.21.15.12:0/37003666 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659800bb20 con 0x7f65a00cdb80 2023-07-08T12:39:05.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.972+0000 7f6597fff700 1 -- 172.21.15.12:0/37003666 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6598012430 con 0x7f65a00cdb80 2023-07-08T12:39:05.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.973+0000 7f65a7910700 1 -- 172.21.15.12:0/37003666 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65a00cdb80 msgr2=0x7f65a00cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.973+0000 7f65a7910700 1 --2- 172.21.15.12:0/37003666 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65a00cdb80 0x7f65a00cbe10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6598009d90 tx=0x7f659800b330 comp rx=0 tx=0).stop 2023-07-08T12:39:05.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.973+0000 7f65a7910700 1 -- 172.21.15.12:0/37003666 shutdown_connections 2023-07-08T12:39:05.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.973+0000 7f65a7910700 1 --2- 172.21.15.12:0/37003666 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65a00cdb80 0x7f65a00cbe10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.973+0000 7f65a7910700 1 --2- 172.21.15.12:0/37003666 >> v2:172.21.15.160:3300/0 conn(0x7f65a00aeec0 0x7f65a006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.973+0000 7f65a7910700 1 -- 172.21.15.12:0/37003666 >> 172.21.15.12:0/37003666 conn(0x7f65a0098b50 msgr2=0x7f65a006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:05.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.973+0000 7f65a7910700 1 -- 172.21.15.12:0/37003666 shutdown_connections 2023-07-08T12:39:05.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.973+0000 7f65a7910700 1 -- 172.21.15.12:0/37003666 wait complete. 2023-07-08T12:39:05.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.974+0000 7f65a7910700 1 Processor -- start 2023-07-08T12:39:05.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.974+0000 7f65a7910700 1 -- start start 2023-07-08T12:39:05.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a7910700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a006ebe0 0x7f65a0158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a7910700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a00aeec0 0x7f65a00b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a7910700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65a00cdb80 0x7f65a014ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a7910700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0146930 con 0x7f65a00aeec0 2023-07-08T12:39:05.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a7910700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a014a3b0 con 0x7f65a006ebe0 2023-07-08T12:39:05.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a7910700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0159350 con 0x7f65a00cdb80 2023-07-08T12:39:05.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a4eab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a00aeec0 0x7f65a00b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a4eab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a00aeec0 0x7f65a00b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50844/0 (socket says 172.21.15.12:50844) 2023-07-08T12:39:05.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a4eab700 1 -- 172.21.15.12:0/364813447 learned_addr learned my addr 172.21.15.12:0/364813447 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:05.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a5ead700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65a00cdb80 0x7f65a014ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a56ac700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a006ebe0 0x7f65a0158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a4eab700 1 -- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65a00cdb80 msgr2=0x7f65a014ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a4eab700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65a00cdb80 0x7f65a014ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a4eab700 1 -- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a006ebe0 msgr2=0x7f65a0158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:05.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a4eab700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a006ebe0 0x7f65a0158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:05.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a4eab700 1 -- 172.21.15.12:0/364813447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6590009430 con 0x7f65a00aeec0 2023-07-08T12:39:05.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a56ac700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a006ebe0 0x7f65a0158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:05.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.975+0000 7f65a4eab700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a00aeec0 0x7f65a00b2660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f659c00c520 tx=0x7f659c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:05.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.976+0000 7f6595ffb700 1 -- 172.21.15.12:0/364813447 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c005d70 con 0x7f65a00aeec0 2023-07-08T12:39:05.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.976+0000 7f6595ffb700 1 -- 172.21.15.12:0/364813447 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c0070d0 con 0x7f65a00aeec0 2023-07-08T12:39:05.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.976+0000 7f65a7910700 1 -- 172.21.15.12:0/364813447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6598009a70 con 0x7f65a00aeec0 2023-07-08T12:39:05.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.976+0000 7f6595ffb700 1 -- 172.21.15.12:0/364813447 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c007440 con 0x7f65a00aeec0 2023-07-08T12:39:05.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.976+0000 7f65a7910700 1 -- 172.21.15.12:0/364813447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a00b5070 con 0x7f65a00aeec0 2023-07-08T12:39:05.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.977+0000 7f65a7910700 1 -- 172.21.15.12:0/364813447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a004ea60 con 0x7f65a00aeec0 2023-07-08T12:39:05.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.977+0000 7f6595ffb700 1 -- 172.21.15.12:0/364813447 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f659c0075a0 con 0x7f65a00aeec0 2023-07-08T12:39:05.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.978+0000 7f6595ffb700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f658406ad10 0x7f658406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:05.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.978+0000 7f6595ffb700 1 -- 172.21.15.12:0/364813447 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f659c088c80 con 0x7f65a00aeec0 2023-07-08T12:39:05.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.978+0000 7f65a56ac700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f658406ad10 0x7f658406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:05.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.978+0000 7f65a56ac700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f658406ad10 0x7f658406d1c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6590009da0 tx=0x7f6590008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:05.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:05.982+0000 7f6595ffb700 1 -- 172.21.15.12:0/364813447 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f659c0539a0 con 0x7f65a00aeec0 2023-07-08T12:39:06.130 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:39:06.132 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:39:06.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.188+0000 7f65a7910700 1 -- 172.21.15.12:0/364813447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f65a00ccde0 con 0x7f65a00aeec0 2023-07-08T12:39:06.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.189+0000 7f6595ffb700 1 -- 172.21.15.12:0/364813447 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f659c056fc0 con 0x7f65a00aeec0 2023-07-08T12:39:06.191 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:06.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.191+0000 7f65a7910700 1 -- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f658406ad10 msgr2=0x7f658406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:06.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.191+0000 7f65a7910700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f658406ad10 0x7f658406d1c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6590009da0 tx=0x7f6590008040 comp rx=0 tx=0).stop 2023-07-08T12:39:06.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.191+0000 7f65a7910700 1 -- 172.21.15.12:0/364813447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a00aeec0 msgr2=0x7f65a00b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:06.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.191+0000 7f65a7910700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a00aeec0 0x7f65a00b2660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f659c00c520 tx=0x7f659c00c830 comp rx=0 tx=0).stop 2023-07-08T12:39:06.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.191+0000 7f65a7910700 1 -- 172.21.15.12:0/364813447 shutdown_connections 2023-07-08T12:39:06.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.191+0000 7f65a7910700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f658406ad10 0x7f658406d1c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.191+0000 7f65a7910700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65a00cdb80 0x7f65a014ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.191+0000 7f65a7910700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65a006ebe0 0x7f65a0158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.191+0000 7f65a7910700 1 --2- 172.21.15.12:0/364813447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a00aeec0 0x7f65a00b2660 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.191+0000 7f65a7910700 1 -- 172.21.15.12:0/364813447 >> 172.21.15.12:0/364813447 conn(0x7f65a0098b50 msgr2=0x7f65a006d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:06.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.191+0000 7f65a7910700 1 -- 172.21.15.12:0/364813447 shutdown_connections 2023-07-08T12:39:06.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.191+0000 7f65a7910700 1 -- 172.21.15.12:0/364813447 wait complete. 2023-07-08T12:39:06.205 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-08T12:39:06.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.369+0000 7f3a8ae70700 1 Processor -- start 2023-07-08T12:39:06.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.369+0000 7f3a8ae70700 1 -- start start 2023-07-08T12:39:06.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a8ae70700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a84160790 0x7f3a840cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a8ae70700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8415c890 con 0x7f3a84159c30 2023-07-08T12:39:06.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a8ae70700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8406f620 con 0x7f3a84069f30 2023-07-08T12:39:06.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a8ae70700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8415c730 con 0x7f3a84160790 2023-07-08T12:39:06.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a88c0c700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3a84159c30 0x7f3a8406ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38246/0 (socket says 172.21.15.12:38246) 2023-07-08T12:39:06.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a88c0c700 1 -- 172.21.15.12:0/876916296 learned_addr learned my addr 172.21.15.12:0/876916296 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:06.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a83fff700 1 --2- 172.21.15.12:0/876916296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a84160790 0x7f3a840cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057269364 0 0) 0x7f3a8415c890 con 0x7f3a84159c30 2023-07-08T12:39:06.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a740035b0 con 0x7f3a84159c30 2023-07-08T12:39:06.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39064894 0 0) 0x7f3a8406f620 con 0x7f3a84069f30 2023-07-08T12:39:06.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a8415c890 con 0x7f3a84069f30 2023-07-08T12:39:06.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3025574662 0 0) 0x7f3a740035b0 con 0x7f3a84159c30 2023-07-08T12:39:06.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a84160790 msgr2=0x7f3a840cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:06.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 --2- 172.21.15.12:0/876916296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a84160790 0x7f3a840cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 >> v1:172.21.15.160:6789/0 conn(0x7f3a84069f30 legacy=0x7f3a8415cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:06.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a84160c90 con 0x7f3a84159c30 2023-07-08T12:39:06.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f3a78001670 con 0x7f3a84159c30 2023-07-08T12:39:06.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 >> v1:172.21.15.12:6789/0 conn(0x7f3a84159c30 legacy=0x7f3a8406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:06.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 --2- 172.21.15.12:0/876916296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a84160790 0x7f3a74003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 --2- 172.21.15.12:0/876916296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a74004170 0x7f3a74004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 --2- 172.21.15.12:0/876916296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a74004d90 0x7f3a74005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8415c730 con 0x7f3a74004170 2023-07-08T12:39:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7400bb20 con 0x7f3a74004d90 2023-07-08T12:39:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.370+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7400bce0 con 0x7f3a84160790 2023-07-08T12:39:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.371+0000 7f3a83fff700 1 --2- 172.21.15.12:0/876916296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a74004170 0x7f3a74004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.371+0000 7f3a88c0c700 1 --2- 172.21.15.12:0/876916296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a84160790 0x7f3a74003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.371+0000 7f3a8940d700 1 --2- 172.21.15.12:0/876916296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a74004d90 0x7f3a74005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.371+0000 7f3a83fff700 1 -- 172.21.15.12:0/876916296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a84160790 msgr2=0x7f3a74003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:06.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.371+0000 7f3a83fff700 1 --2- 172.21.15.12:0/876916296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a84160790 0x7f3a74003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.371+0000 7f3a83fff700 1 -- 172.21.15.12:0/876916296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a74004d90 msgr2=0x7f3a74005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:06.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.371+0000 7f3a83fff700 1 --2- 172.21.15.12:0/876916296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a74004d90 0x7f3a74005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.371+0000 7f3a83fff700 1 -- 172.21.15.12:0/876916296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a740035b0 con 0x7f3a74004170 2023-07-08T12:39:06.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.371+0000 7f3a83fff700 1 --2- 172.21.15.12:0/876916296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a74004170 0x7f3a74004850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009b30 tx=0x7f3a74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec091da5e4d2483c server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:06.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.371+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7000e6c0 con 0x7f3a74004170 2023-07-08T12:39:06.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.371+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7000e820 con 0x7f3a74004170 2023-07-08T12:39:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.371+0000 7f3a837fe700 1 -- 172.21.15.12:0/876916296 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7000eb90 con 0x7f3a74004170 2023-07-08T12:39:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.372+0000 7f3a8ae70700 1 -- 172.21.15.12:0/876916296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a74004170 msgr2=0x7f3a74004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.372+0000 7f3a8ae70700 1 --2- 172.21.15.12:0/876916296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a74004170 0x7f3a74004850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009b30 tx=0x7f3a74005880 comp rx=0 tx=0).stop 2023-07-08T12:39:06.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.372+0000 7f3a8940d700 1 -- 172.21.15.12:0/876916296 reap_dead start 2023-07-08T12:39:06.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.372+0000 7f3a8ae70700 1 -- 172.21.15.12:0/876916296 shutdown_connections 2023-07-08T12:39:06.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.372+0000 7f3a8ae70700 1 -- 172.21.15.12:0/876916296 >> 172.21.15.12:0/876916296 conn(0x7f3a840cc290 msgr2=0x7f3a840cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:06.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.372+0000 7f3a8ae70700 1 -- 172.21.15.12:0/876916296 shutdown_connections 2023-07-08T12:39:06.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.372+0000 7f3a8ae70700 1 -- 172.21.15.12:0/876916296 wait complete. 2023-07-08T12:39:06.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.373+0000 7f3a8ae70700 1 Processor -- start 2023-07-08T12:39:06.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.373+0000 7f3a8ae70700 1 -- start start 2023-07-08T12:39:06.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8ae70700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a74004170 0x7f3a840ac410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8ae70700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a840b08c0 0x7f3a841f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8ae70700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a84154680 0x7f3a8415de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8ae70700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a841f2c70 con 0x7f3a84154680 2023-07-08T12:39:06.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8ae70700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a840ac950 con 0x7f3a840b08c0 2023-07-08T12:39:06.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8ae70700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a840b0430 con 0x7f3a74004170 2023-07-08T12:39:06.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8940d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a84154680 0x7f3a8415de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8940d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a84154680 0x7f3a8415de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50866/0 (socket says 172.21.15.12:50866) 2023-07-08T12:39:06.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8940d700 1 -- 172.21.15.12:0/1286618706 learned_addr learned my addr 172.21.15.12:0/1286618706 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:06.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a88c0c700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a74004170 0x7f3a840ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a83fff700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a840b08c0 0x7f3a841f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8940d700 1 -- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a74004170 msgr2=0x7f3a840ac410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:06.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8940d700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a74004170 0x7f3a840ac410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8940d700 1 -- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a840b08c0 msgr2=0x7f3a841f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8940d700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a840b08c0 0x7f3a841f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8940d700 1 -- 172.21.15.12:0/1286618706 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a840afb00 con 0x7f3a84154680 2023-07-08T12:39:06.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.374+0000 7f3a8940d700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a84154680 0x7f3a8415de70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3a840d08e0 tx=0x7f3a7c00d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:06.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.375+0000 7f3a817fa700 1 -- 172.21.15.12:0/1286618706 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c00db40 con 0x7f3a84154680 2023-07-08T12:39:06.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.375+0000 7f3a817fa700 1 -- 172.21.15.12:0/1286618706 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c00dca0 con 0x7f3a84154680 2023-07-08T12:39:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.375+0000 7f3a817fa700 1 -- 172.21.15.12:0/1286618706 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c003a00 con 0x7f3a84154680 2023-07-08T12:39:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.375+0000 7f3a8ae70700 1 -- 172.21.15.12:0/1286618706 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a840afde0 con 0x7f3a84154680 2023-07-08T12:39:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.375+0000 7f3a8ae70700 1 -- 172.21.15.12:0/1286618706 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a840b0d30 con 0x7f3a84154680 2023-07-08T12:39:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.376+0000 7f3a8ae70700 1 -- 172.21.15.12:0/1286618706 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a840b05a0 con 0x7f3a84154680 2023-07-08T12:39:06.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.376+0000 7f3a817fa700 1 -- 172.21.15.12:0/1286618706 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a7c0050d0 con 0x7f3a84154680 2023-07-08T12:39:06.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.376+0000 7f3a817fa700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3a6806ace0 0x7f3a6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.377+0000 7f3a817fa700 1 -- 172.21.15.12:0/1286618706 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3a7c089cc0 con 0x7f3a84154680 2023-07-08T12:39:06.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.377+0000 7f3a88c0c700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3a6806ace0 0x7f3a6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.377+0000 7f3a88c0c700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3a6806ace0 0x7f3a6806d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a780017a0 tx=0x7f3a780015d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:06.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.381+0000 7f3a817fa700 1 -- 172.21.15.12:0/1286618706 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3a7c090080 con 0x7f3a84154680 2023-07-08T12:39:06.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.596+0000 7f3a8ae70700 1 -- 172.21.15.12:0/1286618706 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3a8404ea60 con 0x7f3a84154680 2023-07-08T12:39:06.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.596+0000 7f3a817fa700 1 -- 172.21.15.12:0/1286618706 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3a7c0549a0 con 0x7f3a84154680 2023-07-08T12:39:06.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.598+0000 7f3a8ae70700 1 -- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3a6806ace0 msgr2=0x7f3a6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:06.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.598+0000 7f3a8ae70700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3a6806ace0 0x7f3a6806d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a780017a0 tx=0x7f3a780015d0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.598+0000 7f3a8ae70700 1 -- 172.21.15.12:0/1286618706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a84154680 msgr2=0x7f3a8415de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:06.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.598+0000 7f3a8ae70700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a84154680 0x7f3a8415de70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3a840d08e0 tx=0x7f3a7c00d660 comp rx=0 tx=0).stop 2023-07-08T12:39:06.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.598+0000 7f3a8ae70700 1 -- 172.21.15.12:0/1286618706 shutdown_connections 2023-07-08T12:39:06.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.598+0000 7f3a8ae70700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f3a6806ace0 0x7f3a6806d190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.598+0000 7f3a8ae70700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a84154680 0x7f3a8415de70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.598+0000 7f3a8ae70700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a74004170 0x7f3a840ac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.598+0000 7f3a8ae70700 1 --2- 172.21.15.12:0/1286618706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a840b08c0 0x7f3a841f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.598+0000 7f3a8ae70700 1 -- 172.21.15.12:0/1286618706 >> 172.21.15.12:0/1286618706 conn(0x7f3a840cc290 msgr2=0x7f3a840714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:06.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.598+0000 7f3a8ae70700 1 -- 172.21.15.12:0/1286618706 shutdown_connections 2023-07-08T12:39:06.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.598+0000 7f3a8ae70700 1 -- 172.21.15.12:0/1286618706 wait complete. 2023-07-08T12:39:06.604 INFO:teuthology.orchestra.run.smithi012.stderr:listed 0 entries 2023-07-08T12:39:06.613 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-08T12:39:06.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:06.614 DEBUG:teuthology.orchestra.run.smithi012:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-07-08T12:39:06.657 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2023-07-08T12:39:06.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:06.659 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-08T12:39:06.715 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:39:06.716 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:39:06.737 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:39:06.737 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:39:06.797 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:39:06.798 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:39:06.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:39:06.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.982+0000 7f3563d29700 1 Processor -- start 2023-07-08T12:39:06.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.982+0000 7f3563d29700 1 -- start start 2023-07-08T12:39:06.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.983+0000 7f3563d29700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f355c06a640 0x7f355c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.983+0000 7f3563d29700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355c15c890 con 0x7f355c160790 2023-07-08T12:39:06.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.983+0000 7f3563d29700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355c06f620 con 0x7f355c159c30 2023-07-08T12:39:06.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.983+0000 7f3563d29700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c15c730 con 0x7f355c06a640 2023-07-08T12:39:06.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.983+0000 7f35612c4700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f355c160790 0x7f355c069f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38268/0 (socket says 172.21.15.12:38268) 2023-07-08T12:39:06.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.983+0000 7f35612c4700 1 -- 172.21.15.12:0/727822282 learned_addr learned my addr 172.21.15.12:0/727822282 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:06.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.983+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008486591 0 0) 0x7f355c15c890 con 0x7f355c160790 2023-07-08T12:39:06.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.983+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35480035b0 con 0x7f355c160790 2023-07-08T12:39:06.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.983+0000 7f35622c6700 1 --2- 172.21.15.12:0/727822282 >> v2:172.21.15.160:3300/0 conn(0x7f355c06a640 0x7f355c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2305097110 0 0) 0x7f35480035b0 con 0x7f355c160790 2023-07-08T12:39:06.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 >> v2:172.21.15.160:3300/0 conn(0x7f355c06a640 msgr2=0x7f355c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:06.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3560ac3700 1 --2- 172.21.15.12:0/727822282 >> v2:172.21.15.160:3300/0 conn(0x7f355c06a640 0x7f355c0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 >> v1:172.21.15.160:6789/0 conn(0x7f355c159c30 legacy=0x7f355c06ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:06.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c160c90 con 0x7f355c160790 2023-07-08T12:39:06.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f3558001670 con 0x7f355c160790 2023-07-08T12:39:06.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 >> v1:172.21.15.12:6789/0 conn(0x7f355c160790 legacy=0x7f355c069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:06.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3560ac3700 1 --2- 172.21.15.12:0/727822282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3548003a00 0x7f3548003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3560ac3700 1 --2- 172.21.15.12:0/727822282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3548004850 0x7f3548004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3560ac3700 1 --2- 172.21.15.12:0/727822282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3548005200 0x7f354800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c15c730 con 0x7f3548003a00 2023-07-08T12:39:06.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354800c110 con 0x7f3548004850 2023-07-08T12:39:06.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354800c2d0 con 0x7f3548005200 2023-07-08T12:39:06.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3561ac5700 1 --2- 172.21.15.12:0/727822282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3548003a00 0x7f3548003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f35612c4700 1 --2- 172.21.15.12:0/727822282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3548004850 0x7f3548004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f35622c6700 1 --2- 172.21.15.12:0/727822282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3548005200 0x7f354800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3561ac5700 1 -- 172.21.15.12:0/727822282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3548005200 msgr2=0x7f354800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:06.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3561ac5700 1 --2- 172.21.15.12:0/727822282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3548005200 0x7f354800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3561ac5700 1 -- 172.21.15.12:0/727822282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3548004850 msgr2=0x7f3548004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:06.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3561ac5700 1 --2- 172.21.15.12:0/727822282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3548004850 0x7f3548004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f35622c6700 1 -- 172.21.15.12:0/727822282 reap_dead start 2023-07-08T12:39:06.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3561ac5700 1 -- 172.21.15.12:0/727822282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35480035b0 con 0x7f3548003a00 2023-07-08T12:39:06.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.984+0000 7f3561ac5700 1 --2- 172.21.15.12:0/727822282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3548003a00 0x7f3548003e10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f354c0017c0 tx=0x7f3548005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b01b9f0d0db541 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:06.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.985+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0052d0 con 0x7f3548003a00 2023-07-08T12:39:06.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.985+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c005450 con 0x7f3548003a00 2023-07-08T12:39:06.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.985+0000 7f3560ac3700 1 -- 172.21.15.12:0/727822282 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0038c0 con 0x7f3548003a00 2023-07-08T12:39:06.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.985+0000 7f3563d29700 1 -- 172.21.15.12:0/727822282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3548003a00 msgr2=0x7f3548003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:06.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.985+0000 7f3563d29700 1 --2- 172.21.15.12:0/727822282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3548003a00 0x7f3548003e10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f354c0017c0 tx=0x7f3548005880 comp rx=0 tx=0).stop 2023-07-08T12:39:06.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.985+0000 7f3563d29700 1 -- 172.21.15.12:0/727822282 shutdown_connections 2023-07-08T12:39:06.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.985+0000 7f3563d29700 1 --2- 172.21.15.12:0/727822282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3548003a00 0x7f3548003e10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:06.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.985+0000 7f3563d29700 1 -- 172.21.15.12:0/727822282 >> 172.21.15.12:0/727822282 conn(0x7f355c0cc290 msgr2=0x7f355c0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:06.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.985+0000 7f3563d29700 1 -- 172.21.15.12:0/727822282 shutdown_connections 2023-07-08T12:39:06.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.985+0000 7f3563d29700 1 -- 172.21.15.12:0/727822282 wait complete. 2023-07-08T12:39:06.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.986+0000 7f3563d29700 1 Processor -- start 2023-07-08T12:39:06.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.987+0000 7f3563d29700 1 -- start start 2023-07-08T12:39:06.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.987+0000 7f3563d29700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3548003a00 0x7f355c0c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.987+0000 7f3563d29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f355c150f30 0x7f355c0c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.987+0000 7f3563d29700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f355c15e620 0x7f355c15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:06.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.987+0000 7f3563d29700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c14ef80 con 0x7f355c150f30 2023-07-08T12:39:06.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.987+0000 7f3563d29700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c15f000 con 0x7f3548003a00 2023-07-08T12:39:06.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.987+0000 7f3563d29700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0c48c0 con 0x7f355c15e620 2023-07-08T12:39:06.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.987+0000 7f35612c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f355c150f30 0x7f355c0c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.987+0000 7f35612c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f355c150f30 0x7f355c0c4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50884/0 (socket says 172.21.15.12:50884) 2023-07-08T12:39:06.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.987+0000 7f35612c4700 1 -- 172.21.15.12:0/768160058 learned_addr learned my addr 172.21.15.12:0/768160058 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:06.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f3561ac5700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3548003a00 0x7f355c0c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:06.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f35622c6700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f355c15e620 0x7f355c15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f35612c4700 1 -- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f355c15e620 msgr2=0x7f355c15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f35612c4700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f355c15e620 0x7f355c15eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f35612c4700 1 -- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3548003a00 msgr2=0x7f355c0c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f35612c4700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3548003a00 0x7f355c0c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f35612c4700 1 -- 172.21.15.12:0/768160058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c0c4ba0 con 0x7f355c150f30 2023-07-08T12:39:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f3561ac5700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3548003a00 0x7f355c0c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:07.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f35612c4700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f355c150f30 0x7f355c0c4290 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f355c160bc0 tx=0x7f3558007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:07.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f35527fc700 1 -- 172.21.15.12:0/768160058 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355800f5e0 con 0x7f355c150f30 2023-07-08T12:39:07.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f35527fc700 1 -- 172.21.15.12:0/768160058 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355800f780 con 0x7f355c150f30 2023-07-08T12:39:07.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f3563d29700 1 -- 172.21.15.12:0/768160058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c0c4e80 con 0x7f355c150f30 2023-07-08T12:39:07.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f35527fc700 1 -- 172.21.15.12:0/768160058 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355800fb30 con 0x7f355c150f30 2023-07-08T12:39:07.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.988+0000 7f3563d29700 1 -- 172.21.15.12:0/768160058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c0c5160 con 0x7f355c150f30 2023-07-08T12:39:07.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.989+0000 7f3563d29700 1 -- 172.21.15.12:0/768160058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c0c5320 con 0x7f355c150f30 2023-07-08T12:39:07.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.989+0000 7f35527fc700 1 -- 172.21.15.12:0/768160058 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f355800fc90 con 0x7f355c150f30 2023-07-08T12:39:07.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.990+0000 7f35527fc700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f354006ace0 0x7f354006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:07.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.990+0000 7f35527fc700 1 -- 172.21.15.12:0/768160058 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3558089740 con 0x7f355c150f30 2023-07-08T12:39:07.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.990+0000 7f3561ac5700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f354006ace0 0x7f354006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:07.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.991+0000 7f3561ac5700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f354006ace0 0x7f354006d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f354c015040 tx=0x7f354c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:07.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:06.994+0000 7f35527fc700 1 -- 172.21.15.12:0/768160058 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f35580544d0 con 0x7f355c150f30 2023-07-08T12:39:07.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.193+0000 7f3563d29700 1 -- 172.21.15.12:0/768160058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f355c14f0c0 con 0x7f355c150f30 2023-07-08T12:39:07.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.195+0000 7f35527fc700 1 -- 172.21.15.12:0/768160058 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20203 (secure 0 0 0) 0x7f355c14f0c0 con 0x7f355c150f30 2023-07-08T12:39:07.198 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:07.198 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:49.386462+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:39:07.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.197+0000 7f3563d29700 1 -- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f354006ace0 msgr2=0x7f354006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.197+0000 7f3563d29700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f354006ace0 0x7f354006d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f354c015040 tx=0x7f354c009030 comp rx=0 tx=0).stop 2023-07-08T12:39:07.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.197+0000 7f3563d29700 1 -- 172.21.15.12:0/768160058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f355c150f30 msgr2=0x7f355c0c4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.197+0000 7f3563d29700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f355c150f30 0x7f355c0c4290 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f355c160bc0 tx=0x7f3558007570 comp rx=0 tx=0).stop 2023-07-08T12:39:07.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.197+0000 7f3563d29700 1 -- 172.21.15.12:0/768160058 shutdown_connections 2023-07-08T12:39:07.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.197+0000 7f3563d29700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f354006ace0 0x7f354006d190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.197+0000 7f3563d29700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f355c15e620 0x7f355c15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.197+0000 7f3563d29700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3548003a00 0x7f355c0c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.197+0000 7f3563d29700 1 --2- 172.21.15.12:0/768160058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f355c150f30 0x7f355c0c4290 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.197+0000 7f3563d29700 1 -- 172.21.15.12:0/768160058 >> 172.21.15.12:0/768160058 conn(0x7f355c0cc290 msgr2=0x7f355c158d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:07.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.197+0000 7f3563d29700 1 -- 172.21.15.12:0/768160058 shutdown_connections 2023-07-08T12:39:07.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.197+0000 7f3563d29700 1 -- 172.21.15.12:0/768160058 wait complete. 2023-07-08T12:39:07.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-08T12:38:05.660194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-08T12:38:06.920177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-08T12:38:10.592749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-08T12:39:07.214 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:39:07.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.340+0000 7f0587e5d700 1 Processor -- start 2023-07-08T12:39:07.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.340+0000 7f0587e5d700 1 -- start start 2023-07-08T12:39:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.341+0000 7f0587e5d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05800b4590 0x7f05800b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.341+0000 7f0587e5d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f058006bda0 0x7f05800d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.341+0000 7f0587e5d700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05800b4ea0 con 0x7f058006c210 2023-07-08T12:39:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.341+0000 7f0587e5d700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058006b390 con 0x7f058006bda0 2023-07-08T12:39:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.341+0000 7f0587e5d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058015d3d0 con 0x7f05800b4590 2023-07-08T12:39:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.341+0000 7f05853f8700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f058006bda0 0x7f05800d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.341+0000 7f05853f8700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f058006bda0 0x7f05800d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50892/0 (socket says 172.21.15.12:50892) 2023-07-08T12:39:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.341+0000 7f05853f8700 1 -- 172.21.15.12:0/2505482599 learned_addr learned my addr 172.21.15.12:0/2505482599 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.342+0000 7f0585bf9700 1 --2- 172.21.15.12:0/2505482599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05800b4590 0x7f05800b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.342+0000 7f05853f8700 1 -- 172.21.15.12:0/2505482599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05800b4590 msgr2=0x7f05800b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.342+0000 7f05853f8700 1 --2- 172.21.15.12:0/2505482599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05800b4590 0x7f05800b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.342+0000 7f05853f8700 1 -- 172.21.15.12:0/2505482599 >> v1:172.21.15.160:6789/0 conn(0x7f058006c210 legacy=0x7f058015c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.342+0000 7f05853f8700 1 -- 172.21.15.12:0/2505482599 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058015d6b0 con 0x7f058006bda0 2023-07-08T12:39:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.342+0000 7f05853f8700 1 --2- 172.21.15.12:0/2505482599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f058006bda0 0x7f05800d0a10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f057c0098f0 tx=0x7f057c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5976bd531f35d701 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:07.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.342+0000 7f0584bf7700 1 -- 172.21.15.12:0/2505482599 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c00b8a0 con 0x7f058006bda0 2023-07-08T12:39:07.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.342+0000 7f0584bf7700 1 -- 172.21.15.12:0/2505482599 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c00ba00 con 0x7f058006bda0 2023-07-08T12:39:07.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.342+0000 7f0584bf7700 1 -- 172.21.15.12:0/2505482599 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c00bd90 con 0x7f058006bda0 2023-07-08T12:39:07.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.342+0000 7f0587e5d700 1 -- 172.21.15.12:0/2505482599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f058006bda0 msgr2=0x7f05800d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.342+0000 7f0587e5d700 1 --2- 172.21.15.12:0/2505482599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f058006bda0 0x7f05800d0a10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f057c0098f0 tx=0x7f057c009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:07.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.343+0000 7f0587e5d700 1 -- 172.21.15.12:0/2505482599 shutdown_connections 2023-07-08T12:39:07.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.343+0000 7f0587e5d700 1 --2- 172.21.15.12:0/2505482599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05800b4590 0x7f05800b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.343+0000 7f0587e5d700 1 --2- 172.21.15.12:0/2505482599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f058006bda0 0x7f05800d0a10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.343+0000 7f0587e5d700 1 -- 172.21.15.12:0/2505482599 >> 172.21.15.12:0/2505482599 conn(0x7f05800cf590 msgr2=0x7f05800cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:07.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.343+0000 7f0587e5d700 1 -- 172.21.15.12:0/2505482599 shutdown_connections 2023-07-08T12:39:07.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.343+0000 7f0587e5d700 1 -- 172.21.15.12:0/2505482599 wait complete. 2023-07-08T12:39:07.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.344+0000 7f0587e5d700 1 Processor -- start 2023-07-08T12:39:07.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.344+0000 7f0587e5d700 1 -- start start 2023-07-08T12:39:07.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f0587e5d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f058006bda0 0x7f0580147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:07.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f0587e5d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f058006c210 0x7f05800cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:07.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f0587e5d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05800b4590 0x7f05800cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:07.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f0587e5d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800c9660 con 0x7f058006c210 2023-07-08T12:39:07.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f0587e5d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800cca40 con 0x7f05800b4590 2023-07-08T12:39:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f0587e5d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05800ca370 con 0x7f058006bda0 2023-07-08T12:39:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05853f8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f058006c210 0x7f05800cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05853f8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f058006c210 0x7f05800cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50898/0 (socket says 172.21.15.12:50898) 2023-07-08T12:39:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05853f8700 1 -- 172.21.15.12:0/558957632 learned_addr learned my addr 172.21.15.12:0/558957632 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f0585bf9700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f058006bda0 0x7f0580147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05863fa700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05800b4590 0x7f05800cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05853f8700 1 -- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f058006bda0 msgr2=0x7f0580147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05853f8700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f058006bda0 0x7f0580147e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05853f8700 1 -- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05800b4590 msgr2=0x7f05800cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05853f8700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05800b4590 0x7f05800cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05853f8700 1 -- 172.21.15.12:0/558957632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c009550 con 0x7f058006c210 2023-07-08T12:39:07.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05863fa700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05800b4590 0x7f05800cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:07.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05853f8700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f058006c210 0x7f05800cbfc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f057c000c00 tx=0x7f057c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:07.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05767fc700 1 -- 172.21.15.12:0/558957632 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c004380 con 0x7f058006c210 2023-07-08T12:39:07.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05767fc700 1 -- 172.21.15.12:0/558957632 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c01f070 con 0x7f058006c210 2023-07-08T12:39:07.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f0587e5d700 1 -- 172.21.15.12:0/558957632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05800c7390 con 0x7f058006c210 2023-07-08T12:39:07.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f05767fc700 1 -- 172.21.15.12:0/558957632 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c0036a0 con 0x7f058006c210 2023-07-08T12:39:07.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.345+0000 7f0587e5d700 1 -- 172.21.15.12:0/558957632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05801483d0 con 0x7f058006c210 2023-07-08T12:39:07.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.346+0000 7f0587e5d700 1 -- 172.21.15.12:0/558957632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058004ea60 con 0x7f058006c210 2023-07-08T12:39:07.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.347+0000 7f05767fc700 1 -- 172.21.15.12:0/558957632 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f057c021030 con 0x7f058006c210 2023-07-08T12:39:07.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.347+0000 7f05767fc700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f056406ace0 0x7f056406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:07.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.347+0000 7f05767fc700 1 -- 172.21.15.12:0/558957632 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f057c08ab80 con 0x7f058006c210 2023-07-08T12:39:07.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.347+0000 7f0585bf9700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f056406ace0 0x7f056406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:07.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.348+0000 7f0585bf9700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f056406ace0 0x7f056406d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0570006b20 tx=0x7f0570008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:07.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.352+0000 7f05767fc700 1 -- 172.21.15.12:0/558957632 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f057c0558f0 con 0x7f058006c210 2023-07-08T12:39:07.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.549+0000 7f0587e5d700 1 -- 172.21.15.12:0/558957632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05800b5350 con 0x7f058006c210 2023-07-08T12:39:07.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.550+0000 7f05767fc700 1 -- 172.21.15.12:0/558957632 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f057c058f10 con 0x7f058006c210 2023-07-08T12:39:07.553 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:39:07.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.551+0000 7f0587e5d700 1 -- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f056406ace0 msgr2=0x7f056406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.551+0000 7f0587e5d700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f056406ace0 0x7f056406d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0570006b20 tx=0x7f0570008040 comp rx=0 tx=0).stop 2023-07-08T12:39:07.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.551+0000 7f0587e5d700 1 -- 172.21.15.12:0/558957632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f058006c210 msgr2=0x7f05800cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.551+0000 7f0587e5d700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f058006c210 0x7f05800cbfc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f057c000c00 tx=0x7f057c004750 comp rx=0 tx=0).stop 2023-07-08T12:39:07.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.552+0000 7f0587e5d700 1 -- 172.21.15.12:0/558957632 shutdown_connections 2023-07-08T12:39:07.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.552+0000 7f0587e5d700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f056406ace0 0x7f056406d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.552+0000 7f0587e5d700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05800b4590 0x7f05800cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.552+0000 7f0587e5d700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f058006bda0 0x7f0580147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.552+0000 7f0587e5d700 1 --2- 172.21.15.12:0/558957632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f058006c210 0x7f05800cbfc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.552+0000 7f0587e5d700 1 -- 172.21.15.12:0/558957632 >> 172.21.15.12:0/558957632 conn(0x7f05800cf590 msgr2=0x7f05800b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:07.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.552+0000 7f0587e5d700 1 -- 172.21.15.12:0/558957632 shutdown_connections 2023-07-08T12:39:07.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.552+0000 7f0587e5d700 1 -- 172.21.15.12:0/558957632 wait complete. 2023-07-08T12:39:07.566 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:39:07.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.692+0000 7f298c5f0700 1 Processor -- start 2023-07-08T12:39:07.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.692+0000 7f298c5f0700 1 -- start start 2023-07-08T12:39:07.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.693+0000 7f298c5f0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f29840ce560 0x7f29840ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:07.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.693+0000 7f298c5f0700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f298415a6d0 0x7f298415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:07.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.693+0000 7f298c5f0700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2984158130 con 0x7f298406f330 2023-07-08T12:39:07.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.693+0000 7f298c5f0700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298415b730 con 0x7f29840ce560 2023-07-08T12:39:07.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.693+0000 7f298c5f0700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29840cee70 con 0x7f298415a6d0 2023-07-08T12:39:07.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.694+0000 7f298a38c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f29840ce560 0x7f29840ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:07.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.694+0000 7f298a38c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f29840ce560 0x7f29840ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50904/0 (socket says 172.21.15.12:50904) 2023-07-08T12:39:07.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.694+0000 7f298a38c700 1 -- 172.21.15.12:0/1746667648 learned_addr learned my addr 172.21.15.12:0/1746667648 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:07.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.694+0000 7f2989b8b700 1 --2- 172.21.15.12:0/1746667648 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f298415a6d0 0x7f298415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:07.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.694+0000 7f298a38c700 1 -- 172.21.15.12:0/1746667648 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f298415a6d0 msgr2=0x7f298415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:07.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.694+0000 7f298a38c700 1 --2- 172.21.15.12:0/1746667648 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f298415a6d0 0x7f298415ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.694+0000 7f298a38c700 1 -- 172.21.15.12:0/1746667648 >> v1:172.21.15.160:6789/0 conn(0x7f298406f330 legacy=0x7f298406f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:07.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.694+0000 7f298a38c700 1 -- 172.21.15.12:0/1746667648 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298415b450 con 0x7f29840ce560 2023-07-08T12:39:07.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.694+0000 7f298a38c700 1 --2- 172.21.15.12:0/1746667648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29840ce560 0x7f29840ce930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f29800098f0 tx=0x7f2980009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2558d735c1cf3f92 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:07.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.694+0000 7f298938a700 1 -- 172.21.15.12:0/1746667648 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980004230 con 0x7f29840ce560 2023-07-08T12:39:07.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.694+0000 7f298938a700 1 -- 172.21.15.12:0/1746667648 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2980004390 con 0x7f29840ce560 2023-07-08T12:39:07.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.694+0000 7f298938a700 1 -- 172.21.15.12:0/1746667648 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800036a0 con 0x7f29840ce560 2023-07-08T12:39:07.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.695+0000 7f298c5f0700 1 -- 172.21.15.12:0/1746667648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29840ce560 msgr2=0x7f29840ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.695+0000 7f298c5f0700 1 --2- 172.21.15.12:0/1746667648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29840ce560 0x7f29840ce930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f29800098f0 tx=0x7f2980009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:07.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.695+0000 7f298c5f0700 1 -- 172.21.15.12:0/1746667648 shutdown_connections 2023-07-08T12:39:07.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.695+0000 7f298c5f0700 1 --2- 172.21.15.12:0/1746667648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29840ce560 0x7f29840ce930 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.695+0000 7f298c5f0700 1 --2- 172.21.15.12:0/1746667648 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f298415a6d0 0x7f298415ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.695+0000 7f298c5f0700 1 -- 172.21.15.12:0/1746667648 >> 172.21.15.12:0/1746667648 conn(0x7f2984099c60 msgr2=0x7f2984089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:07.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.695+0000 7f298c5f0700 1 -- 172.21.15.12:0/1746667648 shutdown_connections 2023-07-08T12:39:07.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.695+0000 7f298c5f0700 1 -- 172.21.15.12:0/1746667648 wait complete. 2023-07-08T12:39:07.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.696+0000 7f298c5f0700 1 Processor -- start 2023-07-08T12:39:07.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.697+0000 7f298c5f0700 1 -- start start 2023-07-08T12:39:07.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.697+0000 7f298c5f0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f298406f330 0x7f29840b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:07.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.697+0000 7f298c5f0700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29840ce560 0x7f298414fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:07.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298c5f0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f298415a6d0 0x7f29840cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:07.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298c5f0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29840c4810 con 0x7f298406f330 2023-07-08T12:39:07.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298c5f0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29840c4af0 con 0x7f298415a6d0 2023-07-08T12:39:07.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298c5f0700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29840c4dd0 con 0x7f29840ce560 2023-07-08T12:39:07.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298a38c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f298406f330 0x7f29840b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:07.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298a38c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f298406f330 0x7f29840b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50910/0 (socket says 172.21.15.12:50910) 2023-07-08T12:39:07.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298a38c700 1 -- 172.21.15.12:0/3437264598 learned_addr learned my addr 172.21.15.12:0/3437264598 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:07.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298a38c700 1 -- 172.21.15.12:0/3437264598 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29840ce560 msgr2=0x7f298414fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298a38c700 1 --2- 172.21.15.12:0/3437264598 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29840ce560 0x7f298414fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298a38c700 1 -- 172.21.15.12:0/3437264598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f298415a6d0 msgr2=0x7f29840cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298ab8d700 1 --2- 172.21.15.12:0/3437264598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f298415a6d0 0x7f29840cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298a38c700 1 --2- 172.21.15.12:0/3437264598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f298415a6d0 0x7f29840cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298a38c700 1 -- 172.21.15.12:0/3437264598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2980009550 con 0x7f298406f330 2023-07-08T12:39:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298a38c700 1 --2- 172.21.15.12:0/3437264598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f298406f330 0x7f29840b08f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2980009130 tx=0x7f2980004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f2976ffd700 1 -- 172.21.15.12:0/3437264598 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980003e50 con 0x7f298406f330 2023-07-08T12:39:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f2976ffd700 1 -- 172.21.15.12:0/3437264598 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298000b660 con 0x7f298406f330 2023-07-08T12:39:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f2976ffd700 1 -- 172.21.15.12:0/3437264598 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298000ba10 con 0x7f298406f330 2023-07-08T12:39:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298c5f0700 1 -- 172.21.15.12:0/3437264598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29840c50b0 con 0x7f298406f330 2023-07-08T12:39:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.698+0000 7f298c5f0700 1 -- 172.21.15.12:0/3437264598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29840b0e30 con 0x7f298406f330 2023-07-08T12:39:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.699+0000 7f298c5f0700 1 -- 172.21.15.12:0/3437264598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29840c55d0 con 0x7f298406f330 2023-07-08T12:39:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.700+0000 7f2976ffd700 1 -- 172.21.15.12:0/3437264598 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f298000bb70 con 0x7f298406f330 2023-07-08T12:39:07.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.700+0000 7f2976ffd700 1 --2- 172.21.15.12:0/3437264598 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f296806ace0 0x7f296806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:07.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.700+0000 7f2976ffd700 1 -- 172.21.15.12:0/3437264598 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f298008b0a0 con 0x7f298406f330 2023-07-08T12:39:07.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.700+0000 7f2989b8b700 1 --2- 172.21.15.12:0/3437264598 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f296806ace0 0x7f296806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:07.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.701+0000 7f2989b8b700 1 --2- 172.21.15.12:0/3437264598 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f296806ace0 0x7f296806d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f29780097f0 tx=0x7f2978008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:07.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.704+0000 7f2976ffd700 1 -- 172.21.15.12:0/3437264598 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2980055eb0 con 0x7f298406f330 2023-07-08T12:39:07.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.901+0000 7f298c5f0700 1 -- 172.21.15.12:0/3437264598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f298404ea60 con 0x7f298406f330 2023-07-08T12:39:07.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.902+0000 7f2976ffd700 1 -- 172.21.15.12:0/3437264598 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2980010380 con 0x7f298406f330 2023-07-08T12:39:07.904 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:39:07.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.904+0000 7f298c5f0700 1 -- 172.21.15.12:0/3437264598 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f296806ace0 msgr2=0x7f296806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.904+0000 7f298c5f0700 1 --2- 172.21.15.12:0/3437264598 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f296806ace0 0x7f296806d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f29780097f0 tx=0x7f2978008040 comp rx=0 tx=0).stop 2023-07-08T12:39:07.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.904+0000 7f298c5f0700 1 -- 172.21.15.12:0/3437264598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f298406f330 msgr2=0x7f29840b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:07.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.904+0000 7f298c5f0700 1 --2- 172.21.15.12:0/3437264598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f298406f330 0x7f29840b08f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2980009130 tx=0x7f2980004750 comp rx=0 tx=0).stop 2023-07-08T12:39:07.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.904+0000 7f298c5f0700 1 -- 172.21.15.12:0/3437264598 shutdown_connections 2023-07-08T12:39:07.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.904+0000 7f298c5f0700 1 --2- 172.21.15.12:0/3437264598 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f296806ace0 0x7f296806d190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.904+0000 7f298c5f0700 1 --2- 172.21.15.12:0/3437264598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f298415a6d0 0x7f29840cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.904+0000 7f298c5f0700 1 --2- 172.21.15.12:0/3437264598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f298406f330 0x7f29840b08f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.904+0000 7f298c5f0700 1 --2- 172.21.15.12:0/3437264598 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29840ce560 0x7f298414fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:07.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.904+0000 7f298c5f0700 1 -- 172.21.15.12:0/3437264598 >> 172.21.15.12:0/3437264598 conn(0x7f2984099c60 msgr2=0x7f2984159280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:07.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.904+0000 7f298c5f0700 1 -- 172.21.15.12:0/3437264598 shutdown_connections 2023-07-08T12:39:07.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:07.904+0000 7f298c5f0700 1 -- 172.21.15.12:0/3437264598 wait complete. 2023-07-08T12:39:07.918 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-08T12:39:08.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.044+0000 7efd60961700 1 Processor -- start 2023-07-08T12:39:08.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.044+0000 7efd60961700 1 -- start start 2023-07-08T12:39:08.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.044+0000 7efd60961700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efd5c15a190 0x7efd5c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.044+0000 7efd60961700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efd5c0acb30 0x7efd5c163c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd60961700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd5c0acf70 con 0x7efd5c0b12b0 2023-07-08T12:39:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd60961700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5c15a560 con 0x7efd5c15a190 2023-07-08T12:39:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd60961700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5c06d2b0 con 0x7efd5c0acb30 2023-07-08T12:39:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd5a59c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efd5c15a190 0x7efd5c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd5a59c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efd5c15a190 0x7efd5c06e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50918/0 (socket says 172.21.15.12:50918) 2023-07-08T12:39:08.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd59d9b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7efd5c0b12b0 0x7efd5c0ac420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38316/0 (socket says 172.21.15.12:38316) 2023-07-08T12:39:08.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd5a59c700 1 -- 172.21.15.12:0/708236908 learned_addr learned my addr 172.21.15.12:0/708236908 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:08.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd5ad9d700 1 --2- 172.21.15.12:0/708236908 >> v2:172.21.15.160:3300/0 conn(0x7efd5c0acb30 0x7efd5c163c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd5a59c700 1 -- 172.21.15.12:0/708236908 >> v2:172.21.15.160:3300/0 conn(0x7efd5c0acb30 msgr2=0x7efd5c163c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:08.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd5a59c700 1 --2- 172.21.15.12:0/708236908 >> v2:172.21.15.160:3300/0 conn(0x7efd5c0acb30 0x7efd5c163c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd5a59c700 1 -- 172.21.15.12:0/708236908 >> v1:172.21.15.12:6789/0 conn(0x7efd5c0b12b0 legacy=0x7efd5c0ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd5a59c700 1 -- 172.21.15.12:0/708236908 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c06d590 con 0x7efd5c15a190 2023-07-08T12:39:08.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd5959a700 1 -- 172.21.15.12:0/708236908 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148708971 0 0) 0x7efd5c0acf70 con 0x7efd5c0b12b0 2023-07-08T12:39:08.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd5a59c700 1 --2- 172.21.15.12:0/708236908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c15a190 0x7efd5c06e350 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0ca2e0 tx=0x7efd54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd603dd2fd77a76f server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:08.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd5959a700 1 -- 172.21.15.12:0/708236908 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54005e30 con 0x7efd5c15a190 2023-07-08T12:39:08.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd5959a700 1 -- 172.21.15.12:0/708236908 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd54004030 con 0x7efd5c15a190 2023-07-08T12:39:08.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.045+0000 7efd5959a700 1 -- 172.21.15.12:0/708236908 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd540043e0 con 0x7efd5c15a190 2023-07-08T12:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.046+0000 7efd60961700 1 -- 172.21.15.12:0/708236908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c15a190 msgr2=0x7efd5c06e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.046+0000 7efd60961700 1 --2- 172.21.15.12:0/708236908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c15a190 0x7efd5c06e350 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0ca2e0 tx=0x7efd54009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.046+0000 7efd60961700 1 -- 172.21.15.12:0/708236908 shutdown_connections 2023-07-08T12:39:08.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.046+0000 7efd60961700 1 --2- 172.21.15.12:0/708236908 >> v2:172.21.15.160:3300/0 conn(0x7efd5c0acb30 0x7efd5c163c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.046+0000 7efd60961700 1 --2- 172.21.15.12:0/708236908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c15a190 0x7efd5c06e350 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.046+0000 7efd60961700 1 -- 172.21.15.12:0/708236908 >> 172.21.15.12:0/708236908 conn(0x7efd5c098b50 msgr2=0x7efd5c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:08.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.046+0000 7efd60961700 1 -- 172.21.15.12:0/708236908 shutdown_connections 2023-07-08T12:39:08.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.046+0000 7efd60961700 1 -- 172.21.15.12:0/708236908 wait complete. 2023-07-08T12:39:08.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.047+0000 7efd60961700 1 Processor -- start 2023-07-08T12:39:08.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.047+0000 7efd60961700 1 -- start start 2023-07-08T12:39:08.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd60961700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c0acb30 0x7efd5c1f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd60961700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5c0b12b0 0x7efd5c0ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd60961700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd5c15a190 0x7efd5c0adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd60961700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0c6870 con 0x7efd5c0acb30 2023-07-08T12:39:08.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd60961700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0c76d0 con 0x7efd5c15a190 2023-07-08T12:39:08.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd60961700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0c6a50 con 0x7efd5c0b12b0 2023-07-08T12:39:08.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd5a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c0acb30 0x7efd5c1f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd5a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c0acb30 0x7efd5c1f1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50924/0 (socket says 172.21.15.12:50924) 2023-07-08T12:39:08.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd5a59c700 1 -- 172.21.15.12:0/2752488846 learned_addr learned my addr 172.21.15.12:0/2752488846 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:08.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd5ad9d700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd5c15a190 0x7efd5c0adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd59d9b700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5c0b12b0 0x7efd5c0ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd5a59c700 1 -- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5c0b12b0 msgr2=0x7efd5c0ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd5a59c700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5c0b12b0 0x7efd5c0ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd5a59c700 1 -- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd5c15a190 msgr2=0x7efd5c0adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.048+0000 7efd5a59c700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd5c15a190 0x7efd5c0adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.049+0000 7efd5a59c700 1 -- 172.21.15.12:0/2752488846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd54009500 con 0x7efd5c0acb30 2023-07-08T12:39:08.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.049+0000 7efd59d9b700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5c0b12b0 0x7efd5c0ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:08.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.049+0000 7efd5a59c700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c0acb30 0x7efd5c1f1b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efd54009c10 tx=0x7efd54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:08.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.049+0000 7efd52ffd700 1 -- 172.21.15.12:0/2752488846 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54003ce0 con 0x7efd5c0acb30 2023-07-08T12:39:08.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.049+0000 7efd52ffd700 1 -- 172.21.15.12:0/2752488846 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd54003e40 con 0x7efd5c0acb30 2023-07-08T12:39:08.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.049+0000 7efd60961700 1 -- 172.21.15.12:0/2752488846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5c0c6d30 con 0x7efd5c0acb30 2023-07-08T12:39:08.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.049+0000 7efd52ffd700 1 -- 172.21.15.12:0/2752488846 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5401c640 con 0x7efd5c0acb30 2023-07-08T12:39:08.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.049+0000 7efd60961700 1 -- 172.21.15.12:0/2752488846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5c0c72f0 con 0x7efd5c0acb30 2023-07-08T12:39:08.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.050+0000 7efd60961700 1 -- 172.21.15.12:0/2752488846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5c04ea60 con 0x7efd5c0acb30 2023-07-08T12:39:08.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.050+0000 7efd52ffd700 1 -- 172.21.15.12:0/2752488846 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efd5401c7a0 con 0x7efd5c0acb30 2023-07-08T12:39:08.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.051+0000 7efd52ffd700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efd3c06ace0 0x7efd3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.051+0000 7efd52ffd700 1 -- 172.21.15.12:0/2752488846 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efd5408a740 con 0x7efd5c0acb30 2023-07-08T12:39:08.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.051+0000 7efd59d9b700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efd3c06ace0 0x7efd3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.052+0000 7efd59d9b700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efd3c06ace0 0x7efd3c06d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efd44000f60 tx=0x7efd44007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:08.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.055+0000 7efd52ffd700 1 -- 172.21.15.12:0/2752488846 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efd54055550 con 0x7efd5c0acb30 2023-07-08T12:39:08.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.255+0000 7efd60961700 1 -- 172.21.15.12:0/2752488846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd5c066af0 con 0x7efd5c0acb30 2023-07-08T12:39:08.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.255+0000 7efd52ffd700 1 -- 172.21.15.12:0/2752488846 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd54058b70 con 0x7efd5c0acb30 2023-07-08T12:39:08.257 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:08.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.257+0000 7efd60961700 1 -- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efd3c06ace0 msgr2=0x7efd3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.257+0000 7efd60961700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efd3c06ace0 0x7efd3c06d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efd44000f60 tx=0x7efd44007d30 comp rx=0 tx=0).stop 2023-07-08T12:39:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.257+0000 7efd60961700 1 -- 172.21.15.12:0/2752488846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c0acb30 msgr2=0x7efd5c1f1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.257+0000 7efd60961700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c0acb30 0x7efd5c1f1b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efd54009c10 tx=0x7efd54004750 comp rx=0 tx=0).stop 2023-07-08T12:39:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.257+0000 7efd60961700 1 -- 172.21.15.12:0/2752488846 shutdown_connections 2023-07-08T12:39:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.257+0000 7efd60961700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efd3c06ace0 0x7efd3c06d190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.257+0000 7efd60961700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd5c15a190 0x7efd5c0adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.257+0000 7efd60961700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c0acb30 0x7efd5c1f1b40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.257+0000 7efd60961700 1 --2- 172.21.15.12:0/2752488846 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5c0b12b0 0x7efd5c0ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.257+0000 7efd60961700 1 -- 172.21.15.12:0/2752488846 >> 172.21.15.12:0/2752488846 conn(0x7efd5c098b50 msgr2=0x7efd5c0af290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:08.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.257+0000 7efd60961700 1 -- 172.21.15.12:0/2752488846 shutdown_connections 2023-07-08T12:39:08.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.257+0000 7efd60961700 1 -- 172.21.15.12:0/2752488846 wait complete. 2023-07-08T12:39:08.272 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-08T12:39:08.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.398+0000 7f2a4455f700 1 Processor -- start 2023-07-08T12:39:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.398+0000 7f2a4455f700 1 -- start start 2023-07-08T12:39:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.399+0000 7f2a4455f700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2a3c0ce560 0x7f2a3c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.399+0000 7f2a4455f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c15a6d0 0x7f2a3c15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.399+0000 7f2a4455f700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2a3c06f330 0x7f2a3c06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.399+0000 7f2a4455f700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c15b730 con 0x7f2a3c06f330 2023-07-08T12:39:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.399+0000 7f2a4455f700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c158130 con 0x7f2a3c0ce560 2023-07-08T12:39:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.399+0000 7f2a4455f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0cee70 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.399+0000 7f2a42afc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2a3c06f330 0x7f2a3c06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.400+0000 7f2a42afc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2a3c06f330 0x7f2a3c06f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50936/0 (socket says 172.21.15.12:50936) 2023-07-08T12:39:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.400+0000 7f2a42afc700 1 -- 172.21.15.12:0/3066212304 learned_addr learned my addr 172.21.15.12:0/3066212304 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.400+0000 7f2a422fb700 1 --2- 172.21.15.12:0/3066212304 >> v2:172.21.15.160:3300/0 conn(0x7f2a3c0ce560 0x7f2a3c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.400+0000 7f2a41afa700 1 --2- 172.21.15.12:0/3066212304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c15a6d0 0x7f2a3c15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.400+0000 7f2a42afc700 1 -- 172.21.15.12:0/3066212304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c15a6d0 msgr2=0x7f2a3c15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.400+0000 7f2a42afc700 1 --2- 172.21.15.12:0/3066212304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c15a6d0 0x7f2a3c15ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.400+0000 7f2a42afc700 1 -- 172.21.15.12:0/3066212304 >> v2:172.21.15.160:3300/0 conn(0x7f2a3c0ce560 msgr2=0x7f2a3c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.400+0000 7f2a42afc700 1 --2- 172.21.15.12:0/3066212304 >> v2:172.21.15.160:3300/0 conn(0x7f2a3c0ce560 0x7f2a3c0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.400+0000 7f2a42afc700 1 -- 172.21.15.12:0/3066212304 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c15b450 con 0x7f2a3c06f330 2023-07-08T12:39:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.400+0000 7f2a42afc700 1 --2- 172.21.15.12:0/3066212304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c06f330 0x7f2a3c06f7b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0718e0 tx=0x7f2a3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98000a41f572885 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:08.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.400+0000 7f2a412f9700 1 -- 172.21.15.12:0/3066212304 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400e070 con 0x7f2a3c06f330 2023-07-08T12:39:08.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.400+0000 7f2a412f9700 1 -- 172.21.15.12:0/3066212304 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3400b9c0 con 0x7f2a3c06f330 2023-07-08T12:39:08.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.400+0000 7f2a412f9700 1 -- 172.21.15.12:0/3066212304 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400bd70 con 0x7f2a3c06f330 2023-07-08T12:39:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.401+0000 7f2a4455f700 1 -- 172.21.15.12:0/3066212304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c06f330 msgr2=0x7f2a3c06f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.401+0000 7f2a4455f700 1 --2- 172.21.15.12:0/3066212304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c06f330 0x7f2a3c06f7b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0718e0 tx=0x7f2a3400b330 comp rx=0 tx=0).stop 2023-07-08T12:39:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.401+0000 7f2a4455f700 1 -- 172.21.15.12:0/3066212304 shutdown_connections 2023-07-08T12:39:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.401+0000 7f2a4455f700 1 --2- 172.21.15.12:0/3066212304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c06f330 0x7f2a3c06f7b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.401+0000 7f2a4455f700 1 --2- 172.21.15.12:0/3066212304 >> v2:172.21.15.160:3300/0 conn(0x7f2a3c0ce560 0x7f2a3c0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.401+0000 7f2a4455f700 1 --2- 172.21.15.12:0/3066212304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c15a6d0 0x7f2a3c15ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.401+0000 7f2a4455f700 1 -- 172.21.15.12:0/3066212304 >> 172.21.15.12:0/3066212304 conn(0x7f2a3c099c60 msgr2=0x7f2a3c089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.401+0000 7f2a4455f700 1 -- 172.21.15.12:0/3066212304 shutdown_connections 2023-07-08T12:39:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.401+0000 7f2a4455f700 1 -- 172.21.15.12:0/3066212304 wait complete. 2023-07-08T12:39:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.402+0000 7f2a4455f700 1 Processor -- start 2023-07-08T12:39:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.402+0000 7f2a4455f700 1 -- start start 2023-07-08T12:39:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.403+0000 7f2a4455f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c06f330 0x7f2a3c1ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.403+0000 7f2a4455f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a3c0ce560 0x7f2a3c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.403+0000 7f2a4455f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c15a6d0 0x7f2a3c0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.403+0000 7f2a4455f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0b1930 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.403+0000 7f2a4455f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0b1bd0 con 0x7f2a3c0ce560 2023-07-08T12:39:08.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.403+0000 7f2a4455f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0b1eb0 con 0x7f2a3c06f330 2023-07-08T12:39:08.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a42afc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c15a6d0 0x7f2a3c0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a42afc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c15a6d0 0x7f2a3c0b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50942/0 (socket says 172.21.15.12:50942) 2023-07-08T12:39:08.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a42afc700 1 -- 172.21.15.12:0/2944381610 learned_addr learned my addr 172.21.15.12:0/2944381610 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:08.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a41afa700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a3c0ce560 0x7f2a3c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a422fb700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c06f330 0x7f2a3c1ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a42afc700 1 -- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c06f330 msgr2=0x7f2a3c1ef380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a42afc700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c06f330 0x7f2a3c1ef380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a42afc700 1 -- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a3c0ce560 msgr2=0x7f2a3c0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a42afc700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a3c0ce560 0x7f2a3c0b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a42afc700 1 -- 172.21.15.12:0/2944381610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a34009b30 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a41afa700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a3c0ce560 0x7f2a3c0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a42afc700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c15a6d0 0x7f2a3c0b4ff0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400b370 tx=0x7f2a34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a32ffd700 1 -- 172.21.15.12:0/2944381610 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34004250 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a32ffd700 1 -- 172.21.15.12:0/2944381610 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a340043b0 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a32ffd700 1 -- 172.21.15.12:0/2944381610 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34004760 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a4455f700 1 -- 172.21.15.12:0/2944381610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c0b2190 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.404+0000 7f2a4455f700 1 -- 172.21.15.12:0/2944381610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c0b2ed0 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.405+0000 7f2a4455f700 1 -- 172.21.15.12:0/2944381610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c0b26b0 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.406+0000 7f2a32ffd700 1 -- 172.21.15.12:0/2944381610 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a3401b430 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.406+0000 7f2a32ffd700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2a2006ace0 0x7f2a2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.406+0000 7f2a32ffd700 1 -- 172.21.15.12:0/2944381610 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2a3408a920 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.407+0000 7f2a422fb700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2a2006ace0 0x7f2a2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.407+0000 7f2a422fb700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2a2006ace0 0x7f2a2006d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00ba10 tx=0x7f2a2c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.411+0000 7f2a32ffd700 1 -- 172.21.15.12:0/2944381610 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2a3c0b26b0 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.614+0000 7f2a4455f700 1 -- 172.21.15.12:0/2944381610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a3c0b26b0 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.614+0000 7f2a32ffd700 1 -- 172.21.15.12:0/2944381610 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a3401f020 con 0x7f2a3c15a6d0 2023-07-08T12:39:08.617 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-07-08T12:39:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.616+0000 7f2a4455f700 1 -- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2a2006ace0 msgr2=0x7f2a2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.616+0000 7f2a4455f700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2a2006ace0 0x7f2a2006d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00ba10 tx=0x7f2a2c008040 comp rx=0 tx=0).stop 2023-07-08T12:39:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.616+0000 7f2a4455f700 1 -- 172.21.15.12:0/2944381610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c15a6d0 msgr2=0x7f2a3c0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.616+0000 7f2a4455f700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c15a6d0 0x7f2a3c0b4ff0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400b370 tx=0x7f2a34004ad0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.616+0000 7f2a4455f700 1 -- 172.21.15.12:0/2944381610 shutdown_connections 2023-07-08T12:39:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.616+0000 7f2a4455f700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2a2006ace0 0x7f2a2006d190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.616+0000 7f2a4455f700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c15a6d0 0x7f2a3c0b4ff0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.616+0000 7f2a4455f700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c06f330 0x7f2a3c1ef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.616+0000 7f2a4455f700 1 --2- 172.21.15.12:0/2944381610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a3c0ce560 0x7f2a3c0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.616+0000 7f2a4455f700 1 -- 172.21.15.12:0/2944381610 >> 172.21.15.12:0/2944381610 conn(0x7f2a3c099c60 msgr2=0x7f2a3c072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:08.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.616+0000 7f2a4455f700 1 -- 172.21.15.12:0/2944381610 shutdown_connections 2023-07-08T12:39:08.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.616+0000 7f2a4455f700 1 -- 172.21.15.12:0/2944381610 wait complete. 2023-07-08T12:39:08.630 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-08T12:39:08.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.756+0000 7fc68ddd9700 1 Processor -- start 2023-07-08T12:39:08.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.756+0000 7fc68ddd9700 1 -- start start 2023-07-08T12:39:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.757+0000 7fc68ddd9700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6880cfc30 0x7fc68806b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.757+0000 7fc68ddd9700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6880ce300 con 0x7fc68806f7f0 2023-07-08T12:39:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.757+0000 7fc68ddd9700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc68815fea0 con 0x7fc68815fad0 2023-07-08T12:39:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.757+0000 7fc68ddd9700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68806e750 con 0x7fc6880cfc30 2023-07-08T12:39:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.757+0000 7fc687fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc68806f7f0 0x7fc6880ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38346/0 (socket says 172.21.15.12:38346) 2023-07-08T12:39:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.757+0000 7fc687fff700 1 -- 172.21.15.12:0/1599697548 learned_addr learned my addr 172.21.15.12:0/1599697548 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc686ffd700 1 --2- 172.21.15.12:0/1599697548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6880cfc30 0x7fc68806b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423789131 0 0) 0x7fc6880ce300 con 0x7fc68806f7f0 2023-07-08T12:39:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6700035b0 con 0x7fc68806f7f0 2023-07-08T12:39:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1045393182 0 0) 0x7fc6700035b0 con 0x7fc68806f7f0 2023-07-08T12:39:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6880cfc30 msgr2=0x7fc68806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:08.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 --2- 172.21.15.12:0/1599697548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6880cfc30 0x7fc68806b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 >> v1:172.21.15.160:6789/0 conn(0x7fc68815fad0 legacy=0x7fc6880cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68806ea30 con 0x7fc68806f7f0 2023-07-08T12:39:08.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7fc6800019f0 con 0x7fc68806f7f0 2023-07-08T12:39:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 >> v1:172.21.15.12:6789/0 conn(0x7fc68806f7f0 legacy=0x7fc6880ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 --2- 172.21.15.12:0/1599697548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc670003a30 0x7fc670003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 --2- 172.21.15.12:0/1599697548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6880cfc30 0x7fc670004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 --2- 172.21.15.12:0/1599697548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc670004d90 0x7fc670005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68806e750 con 0x7fc670004d90 2023-07-08T12:39:08.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67000ba10 con 0x7fc670003a30 2023-07-08T12:39:08.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67000bba0 con 0x7fc6880cfc30 2023-07-08T12:39:08.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc687fff700 1 --2- 172.21.15.12:0/1599697548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc670004d90 0x7fc670005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc6877fe700 1 --2- 172.21.15.12:0/1599697548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc670003a30 0x7fc670003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.758+0000 7fc686ffd700 1 --2- 172.21.15.12:0/1599697548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6880cfc30 0x7fc670004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc687fff700 1 -- 172.21.15.12:0/1599697548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6880cfc30 msgr2=0x7fc670004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:08.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc687fff700 1 --2- 172.21.15.12:0/1599697548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6880cfc30 0x7fc670004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc687fff700 1 -- 172.21.15.12:0/1599697548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc670003a30 msgr2=0x7fc670003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:08.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc687fff700 1 --2- 172.21.15.12:0/1599697548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc670003a30 0x7fc670003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc687fff700 1 -- 172.21.15.12:0/1599697548 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6700035b0 con 0x7fc670004d90 2023-07-08T12:39:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc687fff700 1 --2- 172.21.15.12:0/1599697548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc670004d90 0x7fc670005200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc68000a830 tx=0x7fc670005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c6a078438bdca0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680012600 con 0x7fc670004d90 2023-07-08T12:39:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc680012780 con 0x7fc670004d90 2023-07-08T12:39:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc6867fc700 1 -- 172.21.15.12:0/1599697548 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680012b30 con 0x7fc670004d90 2023-07-08T12:39:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1599697548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc670004d90 msgr2=0x7fc670005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc68ddd9700 1 --2- 172.21.15.12:0/1599697548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc670004d90 0x7fc670005200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc68000a830 tx=0x7fc670005880 comp rx=0 tx=0).stop 2023-07-08T12:39:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc687fff700 1 -- 172.21.15.12:0/1599697548 reap_dead start 2023-07-08T12:39:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1599697548 shutdown_connections 2023-07-08T12:39:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1599697548 >> 172.21.15.12:0/1599697548 conn(0x7fc688099c60 msgr2=0x7fc688089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1599697548 shutdown_connections 2023-07-08T12:39:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.759+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1599697548 wait complete. 2023-07-08T12:39:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.761+0000 7fc68ddd9700 1 Processor -- start 2023-07-08T12:39:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.761+0000 7fc68ddd9700 1 -- start start 2023-07-08T12:39:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc68ddd9700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc670004d90 0x7fc6880c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc68ddd9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc688153b30 0x7fc6881e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc68ddd9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6880c4e90 0x7fc6880c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc68ddd9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6880c8ae0 con 0x7fc6880c4e90 2023-07-08T12:39:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc68ddd9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6880c5870 con 0x7fc688153b30 2023-07-08T12:39:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc68ddd9700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6881e6150 con 0x7fc670004d90 2023-07-08T12:39:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc687fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6880c4e90 0x7fc6880c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc687fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6880c4e90 0x7fc6880c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50962/0 (socket says 172.21.15.12:50962) 2023-07-08T12:39:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc687fff700 1 -- 172.21.15.12:0/1417197456 learned_addr learned my addr 172.21.15.12:0/1417197456 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc686ffd700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc688153b30 0x7fc6881e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc6877fe700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc670004d90 0x7fc6880c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc687fff700 1 -- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc670004d90 msgr2=0x7fc6880c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc687fff700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc670004d90 0x7fc6880c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc687fff700 1 -- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc688153b30 msgr2=0x7fc6881e5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc687fff700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc688153b30 0x7fc6881e5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc687fff700 1 -- 172.21.15.12:0/1417197456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc688153fa0 con 0x7fc6880c4e90 2023-07-08T12:39:08.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc687fff700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6880c4e90 0x7fc6880c5330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc680012000 tx=0x7fc680001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:08.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc66ffff700 1 -- 172.21.15.12:0/1417197456 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6800019b0 con 0x7fc6880c4e90 2023-07-08T12:39:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc66ffff700 1 -- 172.21.15.12:0/1417197456 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc680012610 con 0x7fc6880c4e90 2023-07-08T12:39:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1417197456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6881e6c20 con 0x7fc6880c4e90 2023-07-08T12:39:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc66ffff700 1 -- 172.21.15.12:0/1417197456 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68001e630 con 0x7fc6880c4e90 2023-07-08T12:39:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.762+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1417197456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6881e6660 con 0x7fc6880c4e90 2023-07-08T12:39:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.763+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1417197456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6880c8c20 con 0x7fc6880c4e90 2023-07-08T12:39:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.764+0000 7fc66ffff700 1 -- 172.21.15.12:0/1417197456 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc680024020 con 0x7fc6880c4e90 2023-07-08T12:39:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.764+0000 7fc66ffff700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc67406ace0 0x7fc67406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.764+0000 7fc66ffff700 1 -- 172.21.15.12:0/1417197456 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc68008bbc0 con 0x7fc6880c4e90 2023-07-08T12:39:08.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.764+0000 7fc6877fe700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc67406ace0 0x7fc67406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:08.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.765+0000 7fc6877fe700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc67406ace0 0x7fc67406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc678000c00 tx=0x7fc678009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:08.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.768+0000 7fc66ffff700 1 -- 172.21.15.12:0/1417197456 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc6880c8c20 con 0x7fc6880c4e90 2023-07-08T12:39:08.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.966+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1417197456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc6880c8c20 con 0x7fc6880c4e90 2023-07-08T12:39:08.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.967+0000 7fc66ffff700 1 -- 172.21.15.12:0/1417197456 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc6880c8c20 con 0x7fc6880c4e90 2023-07-08T12:39:08.969 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:08.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.969+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc67406ace0 msgr2=0x7fc67406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.969+0000 7fc68ddd9700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc67406ace0 0x7fc67406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc678000c00 tx=0x7fc678009040 comp rx=0 tx=0).stop 2023-07-08T12:39:08.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.969+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1417197456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6880c4e90 msgr2=0x7fc6880c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:08.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.969+0000 7fc68ddd9700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6880c4e90 0x7fc6880c5330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc680012000 tx=0x7fc680001d80 comp rx=0 tx=0).stop 2023-07-08T12:39:08.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.969+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1417197456 shutdown_connections 2023-07-08T12:39:08.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.969+0000 7fc68ddd9700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc67406ace0 0x7fc67406d190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.969+0000 7fc68ddd9700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6880c4e90 0x7fc6880c5330 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.969+0000 7fc68ddd9700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc670004d90 0x7fc6880c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.969+0000 7fc68ddd9700 1 --2- 172.21.15.12:0/1417197456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc688153b30 0x7fc6881e5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:08.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.969+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1417197456 >> 172.21.15.12:0/1417197456 conn(0x7fc688099c60 msgr2=0x7fc68815df40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:08.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.969+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1417197456 shutdown_connections 2023-07-08T12:39:08.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:08.969+0000 7fc68ddd9700 1 -- 172.21.15.12:0/1417197456 wait complete. 2023-07-08T12:39:08.983 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-08T12:39:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.107+0000 7fc0f1dc3700 1 Processor -- start 2023-07-08T12:39:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.107+0000 7fc0f1dc3700 1 -- start start 2023-07-08T12:39:09.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0f1dc3700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc0ec0ce1b0 0x7fc0ec0cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0f1dc3700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0ec0b1c70 0x7fc0ec0ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0f1dc3700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ec15d230 con 0x7fc0ec15d6c0 2023-07-08T12:39:09.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0f1dc3700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ec0c9e60 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0f1dc3700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec0cd920 con 0x7fc0ec0b1c70 2023-07-08T12:39:09.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0ebfff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc0ec15d6c0 0x7fc0ec071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38366/0 (socket says 172.21.15.12:38366) 2023-07-08T12:39:09.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0ebfff700 1 -- 172.21.15.12:0/1109886680 learned_addr learned my addr 172.21.15.12:0/1109886680 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:09.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0eb7fe700 1 --2- 172.21.15.12:0/1109886680 >> v2:172.21.15.160:3300/0 conn(0x7fc0ec0ce1b0 0x7fc0ec0cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0eaffd700 1 --2- 172.21.15.12:0/1109886680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0ec0b1c70 0x7fc0ec0ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0ea7fc700 1 -- 172.21.15.12:0/1109886680 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2753667042 0 0) 0x7fc0ec15d230 con 0x7fc0ec15d6c0 2023-07-08T12:39:09.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0ea7fc700 1 -- 172.21.15.12:0/1109886680 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0d80035b0 con 0x7fc0ec15d6c0 2023-07-08T12:39:09.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0ea7fc700 1 -- 172.21.15.12:0/1109886680 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 161059961 0 0) 0x7fc0d80035b0 con 0x7fc0ec15d6c0 2023-07-08T12:39:09.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0eaffd700 1 -- 172.21.15.12:0/1109886680 >> v2:172.21.15.160:3300/0 conn(0x7fc0ec0ce1b0 msgr2=0x7fc0ec0cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:09.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0eaffd700 1 --2- 172.21.15.12:0/1109886680 >> v2:172.21.15.160:3300/0 conn(0x7fc0ec0ce1b0 0x7fc0ec0cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0eaffd700 1 -- 172.21.15.12:0/1109886680 >> v1:172.21.15.12:6789/0 conn(0x7fc0ec15d6c0 legacy=0x7fc0ec071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:09.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.108+0000 7fc0eaffd700 1 -- 172.21.15.12:0/1109886680 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec0ceac0 con 0x7fc0ec0b1c70 2023-07-08T12:39:09.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.109+0000 7fc0eaffd700 1 --2- 172.21.15.12:0/1109886680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0ec0b1c70 0x7fc0ec0ce580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009840 tx=0x7fc0dc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=440914efd057e81d server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:09.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.109+0000 7fc0ea7fc700 1 -- 172.21.15.12:0/1109886680 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e40019f0 con 0x7fc0ec0b1c70 2023-07-08T12:39:09.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.109+0000 7fc0ea7fc700 1 -- 172.21.15.12:0/1109886680 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0dc004030 con 0x7fc0ec0b1c70 2023-07-08T12:39:09.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.109+0000 7fc0ea7fc700 1 -- 172.21.15.12:0/1109886680 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0043c0 con 0x7fc0ec0b1c70 2023-07-08T12:39:09.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.109+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/1109886680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0ec0b1c70 msgr2=0x7fc0ec0ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:09.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.109+0000 7fc0f1dc3700 1 --2- 172.21.15.12:0/1109886680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0ec0b1c70 0x7fc0ec0ce580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009840 tx=0x7fc0dc009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:09.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.109+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/1109886680 shutdown_connections 2023-07-08T12:39:09.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.109+0000 7fc0f1dc3700 1 --2- 172.21.15.12:0/1109886680 >> v2:172.21.15.160:3300/0 conn(0x7fc0ec0ce1b0 0x7fc0ec0cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.109+0000 7fc0f1dc3700 1 --2- 172.21.15.12:0/1109886680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0ec0b1c70 0x7fc0ec0ce580 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.109+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/1109886680 >> 172.21.15.12:0/1109886680 conn(0x7fc0ec0cc290 msgr2=0x7fc0ec0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:09.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.109+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/1109886680 shutdown_connections 2023-07-08T12:39:09.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.109+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/1109886680 wait complete. 2023-07-08T12:39:09.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.111+0000 7fc0f1dc3700 1 Processor -- start 2023-07-08T12:39:09.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.111+0000 7fc0f1dc3700 1 -- start start 2023-07-08T12:39:09.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.111+0000 7fc0f1dc3700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0ec0b1c70 0x7fc0ec164ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.111+0000 7fc0f1dc3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0ec0ce1b0 0x7fc0ec1541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.111+0000 7fc0f1dc3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0ec15d6c0 0x7fc0ec1627f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.111+0000 7fc0f1dc3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec1660b0 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.111+0000 7fc0f1dc3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec1654a0 con 0x7fc0ec15d6c0 2023-07-08T12:39:09.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.111+0000 7fc0f1dc3700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec165780 con 0x7fc0ec0b1c70 2023-07-08T12:39:09.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0eaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0ec0ce1b0 0x7fc0ec1541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0eaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0ec0ce1b0 0x7fc0ec1541e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50974/0 (socket says 172.21.15.12:50974) 2023-07-08T12:39:09.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0eaffd700 1 -- 172.21.15.12:0/3074603905 learned_addr learned my addr 172.21.15.12:0/3074603905 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:09.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0eb7fe700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0ec0b1c70 0x7fc0ec164ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0ebfff700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0ec15d6c0 0x7fc0ec1627f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0eaffd700 1 -- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0ec0b1c70 msgr2=0x7fc0ec164ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0eaffd700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0ec0b1c70 0x7fc0ec164ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0eaffd700 1 -- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0ec15d6c0 msgr2=0x7fc0ec1627f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0eaffd700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0ec15d6c0 0x7fc0ec1627f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0eaffd700 1 -- 172.21.15.12:0/3074603905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc0094a0 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0eb7fe700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0ec0b1c70 0x7fc0ec164ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0eaffd700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0ec0ce1b0 0x7fc0ec1541e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00a010 tx=0x7fc0dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:09.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0e3fff700 1 -- 172.21.15.12:0/3074603905 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc003c40 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0e3fff700 1 -- 172.21.15.12:0/3074603905 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0dc01f070 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0e3fff700 1 -- 172.21.15.12:0/3074603905 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc011630 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/3074603905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ec165a60 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.112+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/3074603905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ec154720 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.113+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/3074603905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ec04ea60 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.114+0000 7fc0e3fff700 1 -- 172.21.15.12:0/3074603905 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc0dc021030 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.114+0000 7fc0e3fff700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc0cc06ace0 0x7fc0cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.114+0000 7fc0e3fff700 1 -- 172.21.15.12:0/3074603905 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc0dc08aa90 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.114+0000 7fc0eb7fe700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc0cc06ace0 0x7fc0cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.115+0000 7fc0eb7fe700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc0cc06ace0 0x7fc0cc06d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009b90 tx=0x7fc0d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:09.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.118+0000 7fc0e3fff700 1 -- 172.21.15.12:0/3074603905 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc0dc0559c0 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.332+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/3074603905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc0ec0b5790 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.332+0000 7fc0e3fff700 1 -- 172.21.15.12:0/3074603905 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc0dc058fe0 con 0x7fc0ec0ce1b0 2023-07-08T12:39:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.334+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc0cc06ace0 msgr2=0x7fc0cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.334+0000 7fc0f1dc3700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc0cc06ace0 0x7fc0cc06d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009b90 tx=0x7fc0d4008040 comp rx=0 tx=0).stop 2023-07-08T12:39:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.334+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/3074603905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0ec0ce1b0 msgr2=0x7fc0ec1541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.334+0000 7fc0f1dc3700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0ec0ce1b0 0x7fc0ec1541e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00a010 tx=0x7fc0dc004750 comp rx=0 tx=0).stop 2023-07-08T12:39:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.334+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/3074603905 shutdown_connections 2023-07-08T12:39:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.334+0000 7fc0f1dc3700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc0cc06ace0 0x7fc0cc06d190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.334+0000 7fc0f1dc3700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0ec15d6c0 0x7fc0ec1627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.334+0000 7fc0f1dc3700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc0ec0b1c70 0x7fc0ec164ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.334+0000 7fc0f1dc3700 1 --2- 172.21.15.12:0/3074603905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc0ec0ce1b0 0x7fc0ec1541e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.334+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/3074603905 >> 172.21.15.12:0/3074603905 conn(0x7fc0ec0cc290 msgr2=0x7fc0ec0cb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.334+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/3074603905 shutdown_connections 2023-07-08T12:39:09.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.334+0000 7fc0f1dc3700 1 -- 172.21.15.12:0/3074603905 wait complete. 2023-07-08T12:39:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:listed 0 entries 2023-07-08T12:39:09.348 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-07-08T12:39:09.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:09.349 DEBUG:teuthology.orchestra.run.smithi160:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-07-08T12:39:09.394 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi160.front.sepia.ceph.com 2023-07-08T12:39:09.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:09.395 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-08T12:39:09.421 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:39:09.422 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:39:09.439 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:39:09.440 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:39:09.496 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:39:09.497 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:39:09.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:39:09.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.716+0000 7f789a435700 1 Processor -- start 2023-07-08T12:39:09.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.717+0000 7f789a435700 1 -- start start 2023-07-08T12:39:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.717+0000 7f789a435700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f789406a3a0 0x7f7894071560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.717+0000 7f789a435700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78940b1de0 con 0x7f789415a370 2023-07-08T12:39:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.717+0000 7f789a435700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78940cb7d0 con 0x7f7894069f30 2023-07-08T12:39:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.717+0000 7f789a435700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789415a740 con 0x7f789406a3a0 2023-07-08T12:39:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.717+0000 7f7893fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f789415a370 0x7f789415e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38376/0 (socket says 172.21.15.12:38376) 2023-07-08T12:39:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.717+0000 7f7893fff700 1 -- 172.21.15.12:0/1371116295 learned_addr learned my addr 172.21.15.12:0/1371116295 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f78989d2700 1 --2- 172.21.15.12:0/1371116295 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f789406a3a0 0x7f7894071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373388760 0 0) 0x7f78940b1de0 con 0x7f789415a370 2023-07-08T12:39:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f787c0035b0 con 0x7f789415a370 2023-07-08T12:39:09.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1058512381 0 0) 0x7f787c0035b0 con 0x7f789415a370 2023-07-08T12:39:09.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f789406a3a0 msgr2=0x7f7894071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:09.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 --2- 172.21.15.12:0/1371116295 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f789406a3a0 0x7f7894071560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 >> v1:172.21.15.160:6789/0 conn(0x7f7894069f30 legacy=0x7f7894070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:09.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940cb4f0 con 0x7f789415a370 2023-07-08T12:39:09.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f7884001670 con 0x7f789415a370 2023-07-08T12:39:09.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 >> v1:172.21.15.12:6789/0 conn(0x7f789415a370 legacy=0x7f789415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:09.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 --2- 172.21.15.12:0/1371116295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f787c003a30 0x7f787c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 --2- 172.21.15.12:0/1371116295 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f789406a3a0 0x7f787c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 --2- 172.21.15.12:0/1371116295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787c004d90 0x7f787c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789415a740 con 0x7f787c003a30 2023-07-08T12:39:09.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c00bac0 con 0x7f787c004d90 2023-07-08T12:39:09.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c00bc80 con 0x7f789406a3a0 2023-07-08T12:39:09.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f7893fff700 1 --2- 172.21.15.12:0/1371116295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f787c003a30 0x7f787c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f78937fe700 1 --2- 172.21.15.12:0/1371116295 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f789406a3a0 0x7f787c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.718+0000 7f78989d2700 1 --2- 172.21.15.12:0/1371116295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787c004d90 0x7f787c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.719+0000 7f7893fff700 1 -- 172.21.15.12:0/1371116295 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f789406a3a0 msgr2=0x7f787c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.719+0000 7f7893fff700 1 --2- 172.21.15.12:0/1371116295 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f789406a3a0 0x7f787c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.719+0000 7f7893fff700 1 -- 172.21.15.12:0/1371116295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787c004d90 msgr2=0x7f787c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.719+0000 7f7893fff700 1 --2- 172.21.15.12:0/1371116295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787c004d90 0x7f787c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.719+0000 7f7893fff700 1 -- 172.21.15.12:0/1371116295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c0035b0 con 0x7f787c003a30 2023-07-08T12:39:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.719+0000 7f78989d2700 1 --2- 172.21.15.12:0/1371116295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787c004d90 0x7f787c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.719+0000 7f7893fff700 1 --2- 172.21.15.12:0/1371116295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f787c003a30 0x7f787c003e00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f788400c2f0 tx=0x7f787c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8a0c4bb0359091 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.719+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788400ce30 con 0x7f787c003a30 2023-07-08T12:39:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.719+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7884005960 con 0x7f787c003a30 2023-07-08T12:39:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.719+0000 7f7892ffd700 1 -- 172.21.15.12:0/1371116295 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884005cf0 con 0x7f787c003a30 2023-07-08T12:39:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.719+0000 7f789a435700 1 -- 172.21.15.12:0/1371116295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f787c003a30 msgr2=0x7f787c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.719+0000 7f789a435700 1 --2- 172.21.15.12:0/1371116295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f787c003a30 0x7f787c003e00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f788400c2f0 tx=0x7f787c005880 comp rx=0 tx=0).stop 2023-07-08T12:39:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.719+0000 7f78989d2700 1 -- 172.21.15.12:0/1371116295 reap_dead start 2023-07-08T12:39:09.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.720+0000 7f789a435700 1 -- 172.21.15.12:0/1371116295 shutdown_connections 2023-07-08T12:39:09.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.720+0000 7f789a435700 1 -- 172.21.15.12:0/1371116295 >> 172.21.15.12:0/1371116295 conn(0x7f78940cb9f0 msgr2=0x7f78940cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:09.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.720+0000 7f789a435700 1 -- 172.21.15.12:0/1371116295 shutdown_connections 2023-07-08T12:39:09.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.720+0000 7f789a435700 1 -- 172.21.15.12:0/1371116295 wait complete. 2023-07-08T12:39:09.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.721+0000 7f789a435700 1 Processor -- start 2023-07-08T12:39:09.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.721+0000 7f789a435700 1 -- start start 2023-07-08T12:39:09.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f789a435700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f787c003a30 0x7f789414b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f789a435700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f789414d450 0x7f78940c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f789a435700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f789414ea50 0x7f789414eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f789a435700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7894147f00 con 0x7f789414d450 2023-07-08T12:39:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f789a435700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789414f410 con 0x7f789414ea50 2023-07-08T12:39:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f789a435700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78940b2b90 con 0x7f787c003a30 2023-07-08T12:39:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f78937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f789414d450 0x7f78940c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f78937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f789414d450 0x7f78940c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50994/0 (socket says 172.21.15.12:50994) 2023-07-08T12:39:09.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f78937fe700 1 -- 172.21.15.12:0/3352432365 learned_addr learned my addr 172.21.15.12:0/3352432365 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f78989d2700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f789414ea50 0x7f789414eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f7893fff700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f787c003a30 0x7f789414b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f78937fe700 1 -- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f787c003a30 msgr2=0x7f789414b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f78937fe700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f787c003a30 0x7f789414b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f78937fe700 1 -- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f789414ea50 msgr2=0x7f789414eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f78937fe700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f789414ea50 0x7f789414eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f78937fe700 1 -- 172.21.15.12:0/3352432365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940b1f80 con 0x7f789414d450 2023-07-08T12:39:09.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f7893fff700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f787c003a30 0x7f789414b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f78937fe700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f789414d450 0x7f78940c4640 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f788800f810 tx=0x7f788800fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.722+0000 7f7890ff9700 1 -- 172.21.15.12:0/3352432365 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880052e0 con 0x7f789414d450 2023-07-08T12:39:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.723+0000 7f7890ff9700 1 -- 172.21.15.12:0/3352432365 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7888005440 con 0x7f789414d450 2023-07-08T12:39:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.723+0000 7f789a435700 1 -- 172.21.15.12:0/3352432365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78940b2260 con 0x7f789414d450 2023-07-08T12:39:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.723+0000 7f7890ff9700 1 -- 172.21.15.12:0/3352432365 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880038c0 con 0x7f789414d450 2023-07-08T12:39:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.723+0000 7f789a435700 1 -- 172.21.15.12:0/3352432365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78940b2820 con 0x7f789414d450 2023-07-08T12:39:09.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.723+0000 7f789a435700 1 -- 172.21.15.12:0/3352432365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78940b3e80 con 0x7f789414d450 2023-07-08T12:39:09.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.724+0000 7f7890ff9700 1 -- 172.21.15.12:0/3352432365 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7888003a20 con 0x7f789414d450 2023-07-08T12:39:09.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.724+0000 7f7890ff9700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f788006ace0 0x7f788006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:09.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.724+0000 7f7890ff9700 1 -- 172.21.15.12:0/3352432365 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7888089650 con 0x7f789414d450 2023-07-08T12:39:09.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.725+0000 7f7893fff700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f788006ace0 0x7f788006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:09.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.725+0000 7f7893fff700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f788006ace0 0x7f788006d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7884008fd0 tx=0x7f7884007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:09.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.729+0000 7f7890ff9700 1 -- 172.21.15.12:0/3352432365 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f78880543d0 con 0x7f789414d450 2023-07-08T12:39:09.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.928+0000 7f789a435700 1 -- 172.21.15.12:0/3352432365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f789404ea60 con 0x7f789414d450 2023-07-08T12:39:09.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.930+0000 7f7890ff9700 1 -- 172.21.15.12:0/3352432365 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20203 (secure 0 0 0) 0x7f78880579f0 con 0x7f789414d450 2023-07-08T12:39:09.932 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:09.933 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:49.386462+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:39:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.932+0000 7f789a435700 1 -- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f788006ace0 msgr2=0x7f788006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.932+0000 7f789a435700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f788006ace0 0x7f788006d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7884008fd0 tx=0x7f7884007000 comp rx=0 tx=0).stop 2023-07-08T12:39:09.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.932+0000 7f789a435700 1 -- 172.21.15.12:0/3352432365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f789414d450 msgr2=0x7f78940c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:09.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.932+0000 7f789a435700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f789414d450 0x7f78940c4640 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f788800f810 tx=0x7f788800fb20 comp rx=0 tx=0).stop 2023-07-08T12:39:09.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.932+0000 7f789a435700 1 -- 172.21.15.12:0/3352432365 shutdown_connections 2023-07-08T12:39:09.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.932+0000 7f789a435700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f788006ace0 0x7f788006d190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.932+0000 7f789a435700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f789414ea50 0x7f789414eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.932+0000 7f789a435700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f787c003a30 0x7f789414b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.932+0000 7f789a435700 1 --2- 172.21.15.12:0/3352432365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f789414d450 0x7f78940c4640 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:09.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.932+0000 7f789a435700 1 -- 172.21.15.12:0/3352432365 >> 172.21.15.12:0/3352432365 conn(0x7f78940cb9f0 msgr2=0x7f78940c71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:09.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.932+0000 7f789a435700 1 -- 172.21.15.12:0/3352432365 shutdown_connections 2023-07-08T12:39:09.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:09.932+0000 7f789a435700 1 -- 172.21.15.12:0/3352432365 wait complete. 2023-07-08T12:39:09.946 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-08T12:38:05.660194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-08T12:38:06.920177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-08T12:38:10.592749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-08T12:39:09.947 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:39:10.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.073+0000 7fd58f13f700 1 Processor -- start 2023-07-08T12:39:10.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.073+0000 7fd58f13f700 1 -- start start 2023-07-08T12:39:10.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.074+0000 7fd58f13f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd588156a10 0x7fd58806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.074+0000 7fd58f13f700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd58815d390 0x7fd5881608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.074+0000 7fd58f13f700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd588160410 con 0x7fd5880cc240 2023-07-08T12:39:10.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.074+0000 7fd58f13f700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd588156de0 con 0x7fd58815d390 2023-07-08T12:39:10.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.074+0000 7fd58f13f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58806ec70 con 0x7fd588156a10 2023-07-08T12:39:10.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.075+0000 7fd58cedb700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd588156a10 0x7fd58806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.075+0000 7fd587fff700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd58815d390 0x7fd5881608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.075+0000 7fd58d6dc700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd5880cc240 0x7fd5880ae440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:47622/0 (socket says 172.21.15.12:47622) 2023-07-08T12:39:10.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.075+0000 7fd58d6dc700 1 -- 172.21.15.12:0/143693549 learned_addr learned my addr 172.21.15.12:0/143693549 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:10.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.075+0000 7fd587fff700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd58815d390 0x7fd5881608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:49744/0 (socket says 172.21.15.12:49744) 2023-07-08T12:39:10.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.075+0000 7fd587fff700 1 -- 172.21.15.12:0/143693549 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd588156a10 msgr2=0x7fd58806e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:10.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.075+0000 7fd587fff700 1 --2- 172.21.15.12:0/143693549 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd588156a10 0x7fd58806e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.075+0000 7fd587fff700 1 -- 172.21.15.12:0/143693549 >> v1:172.21.15.160:6789/0 conn(0x7fd5880cc240 legacy=0x7fd5880ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.075+0000 7fd587fff700 1 -- 172.21.15.12:0/143693549 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58806e6b0 con 0x7fd58815d390 2023-07-08T12:39:10.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.075+0000 7fd587fff700 1 --2- 172.21.15.12:0/143693549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd58815d390 0x7fd5881608a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd574009840 tx=0x7fd574009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c34594710d1d42a server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:10.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.075+0000 7fd5877fe700 1 -- 172.21.15.12:0/143693549 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574011830 con 0x7fd58815d390 2023-07-08T12:39:10.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.075+0000 7fd5877fe700 1 -- 172.21.15.12:0/143693549 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd574011990 con 0x7fd58815d390 2023-07-08T12:39:10.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.075+0000 7fd5877fe700 1 -- 172.21.15.12:0/143693549 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574011d20 con 0x7fd58815d390 2023-07-08T12:39:10.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.076+0000 7fd58f13f700 1 -- 172.21.15.12:0/143693549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd58815d390 msgr2=0x7fd5881608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.076+0000 7fd58f13f700 1 --2- 172.21.15.12:0/143693549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd58815d390 0x7fd5881608a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd574009840 tx=0x7fd574009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:10.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.076+0000 7fd58f13f700 1 -- 172.21.15.12:0/143693549 shutdown_connections 2023-07-08T12:39:10.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.076+0000 7fd58f13f700 1 --2- 172.21.15.12:0/143693549 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd588156a10 0x7fd58806e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.076+0000 7fd58f13f700 1 --2- 172.21.15.12:0/143693549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd58815d390 0x7fd5881608a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.076+0000 7fd58f13f700 1 -- 172.21.15.12:0/143693549 >> 172.21.15.12:0/143693549 conn(0x7fd588098b50 msgr2=0x7fd58806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:10.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.076+0000 7fd58f13f700 1 -- 172.21.15.12:0/143693549 shutdown_connections 2023-07-08T12:39:10.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.076+0000 7fd58f13f700 1 -- 172.21.15.12:0/143693549 wait complete. 2023-07-08T12:39:10.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.077+0000 7fd58f13f700 1 Processor -- start 2023-07-08T12:39:10.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.077+0000 7fd58f13f700 1 -- start start 2023-07-08T12:39:10.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.078+0000 7fd58f13f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5880cc240 0x7fd58815bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.078+0000 7fd58f13f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd588156a10 0x7fd58815c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.078+0000 7fd58f13f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd58815d390 0x7fd5881552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.078+0000 7fd58f13f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58815af10 con 0x7fd5880cc240 2023-07-08T12:39:10.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.078+0000 7fd58f13f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58815ca40 con 0x7fd58815d390 2023-07-08T12:39:10.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.078+0000 7fd58f13f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5881560c0 con 0x7fd588156a10 2023-07-08T12:39:10.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.078+0000 7fd58cedb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5880cc240 0x7fd58815bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.078+0000 7fd58d6dc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd58815d390 0x7fd5881552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.078+0000 7fd58cedb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5880cc240 0x7fd58815bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51006/0 (socket says 172.21.15.12:51006) 2023-07-08T12:39:10.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.078+0000 7fd58cedb700 1 -- 172.21.15.12:0/3985430866 learned_addr learned my addr 172.21.15.12:0/3985430866 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:10.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.078+0000 7fd587fff700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd588156a10 0x7fd58815c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.079+0000 7fd58cedb700 1 -- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd588156a10 msgr2=0x7fd58815c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.079+0000 7fd58cedb700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd588156a10 0x7fd58815c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.079+0000 7fd58cedb700 1 -- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd58815d390 msgr2=0x7fd5881552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.079+0000 7fd58cedb700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd58815d390 0x7fd5881552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.079+0000 7fd58cedb700 1 -- 172.21.15.12:0/3985430866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5740094a0 con 0x7fd5880cc240 2023-07-08T12:39:10.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.079+0000 7fd58d6dc700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd58815d390 0x7fd5881552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:39:10.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.079+0000 7fd58cedb700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5880cc240 0x7fd58815bc70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00e990 tx=0x7fd57c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:10.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.079+0000 7fd5857fa700 1 -- 172.21.15.12:0/3985430866 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0096f0 con 0x7fd5880cc240 2023-07-08T12:39:10.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.079+0000 7fd5857fa700 1 -- 172.21.15.12:0/3985430866 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c00c690 con 0x7fd5880cc240 2023-07-08T12:39:10.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.079+0000 7fd58f13f700 1 -- 172.21.15.12:0/3985430866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58815b190 con 0x7fd5880cc240 2023-07-08T12:39:10.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.079+0000 7fd5857fa700 1 -- 172.21.15.12:0/3985430866 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00ca00 con 0x7fd5880cc240 2023-07-08T12:39:10.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.079+0000 7fd58f13f700 1 -- 172.21.15.12:0/3985430866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58815b750 con 0x7fd5880cc240 2023-07-08T12:39:10.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.080+0000 7fd58f13f700 1 -- 172.21.15.12:0/3985430866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58815b940 con 0x7fd5880cc240 2023-07-08T12:39:10.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.080+0000 7fd5857fa700 1 -- 172.21.15.12:0/3985430866 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd57c00cb60 con 0x7fd5880cc240 2023-07-08T12:39:10.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.081+0000 7fd5857fa700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd56c06ace0 0x7fd56c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.081+0000 7fd5857fa700 1 -- 172.21.15.12:0/3985430866 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd57c089b70 con 0x7fd5880cc240 2023-07-08T12:39:10.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.081+0000 7fd587fff700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd56c06ace0 0x7fd56c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.082+0000 7fd587fff700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd56c06ace0 0x7fd56c06d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd574000c00 tx=0x7fd574008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:10.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.086+0000 7fd5857fa700 1 -- 172.21.15.12:0/3985430866 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd57c054900 con 0x7fd5880cc240 2023-07-08T12:39:10.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.284+0000 7fd58f13f700 1 -- 172.21.15.12:0/3985430866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd58804ea60 con 0x7fd5880cc240 2023-07-08T12:39:10.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.285+0000 7fd5857fa700 1 -- 172.21.15.12:0/3985430866 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd57c057f20 con 0x7fd5880cc240 2023-07-08T12:39:10.288 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:39:10.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.287+0000 7fd58f13f700 1 -- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd56c06ace0 msgr2=0x7fd56c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.287+0000 7fd58f13f700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd56c06ace0 0x7fd56c06d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd574000c00 tx=0x7fd574008030 comp rx=0 tx=0).stop 2023-07-08T12:39:10.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.287+0000 7fd58f13f700 1 -- 172.21.15.12:0/3985430866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5880cc240 msgr2=0x7fd58815bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.287+0000 7fd58f13f700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5880cc240 0x7fd58815bc70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00e990 tx=0x7fd57c00eca0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.287+0000 7fd58f13f700 1 -- 172.21.15.12:0/3985430866 shutdown_connections 2023-07-08T12:39:10.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.287+0000 7fd58f13f700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd56c06ace0 0x7fd56c06d190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.287+0000 7fd58f13f700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd58815d390 0x7fd5881552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.287+0000 7fd58f13f700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5880cc240 0x7fd58815bc70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.287+0000 7fd58f13f700 1 --2- 172.21.15.12:0/3985430866 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd588156a10 0x7fd58815c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.287+0000 7fd58f13f700 1 -- 172.21.15.12:0/3985430866 >> 172.21.15.12:0/3985430866 conn(0x7fd588098b50 msgr2=0x7fd5880af680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:10.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.287+0000 7fd58f13f700 1 -- 172.21.15.12:0/3985430866 shutdown_connections 2023-07-08T12:39:10.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.287+0000 7fd58f13f700 1 -- 172.21.15.12:0/3985430866 wait complete. 2023-07-08T12:39:10.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:39:10.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.426+0000 7fa8109f9700 1 Processor -- start 2023-07-08T12:39:10.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.426+0000 7fa8109f9700 1 -- start start 2023-07-08T12:39:10.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.427+0000 7fa8109f9700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa80c156a10 0x7fa80c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.427+0000 7fa8109f9700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa80c0cc240 0x7fa80c0cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.427+0000 7fa8109f9700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa80c0ccbe0 con 0x7fa80c15d390 2023-07-08T12:39:10.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.427+0000 7fa8109f9700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80c160410 con 0x7fa80c156a10 2023-07-08T12:39:10.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa8109f9700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80c156de0 con 0x7fa80c0cc240 2023-07-08T12:39:10.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa80a59c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa80c156a10 0x7fa80c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa809d9b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa80c15d390 0x7fa80c1608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38412/0 (socket says 172.21.15.12:38412) 2023-07-08T12:39:10.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa809d9b700 1 -- 172.21.15.12:0/3658129420 learned_addr learned my addr 172.21.15.12:0/3658129420 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa80ad9d700 1 --2- 172.21.15.12:0/3658129420 >> v2:172.21.15.160:3300/0 conn(0x7fa80c0cc240 0x7fa80c0cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa80a59c700 1 -- 172.21.15.12:0/3658129420 >> v2:172.21.15.160:3300/0 conn(0x7fa80c0cc240 msgr2=0x7fa80c0cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa80a59c700 1 --2- 172.21.15.12:0/3658129420 >> v2:172.21.15.160:3300/0 conn(0x7fa80c0cc240 0x7fa80c0cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa80a59c700 1 -- 172.21.15.12:0/3658129420 >> v1:172.21.15.12:6789/0 conn(0x7fa80c15d390 legacy=0x7fa80c1608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa80959a700 1 -- 172.21.15.12:0/3658129420 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018969175 0 0) 0x7fa80c0ccbe0 con 0x7fa80c15d390 2023-07-08T12:39:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa80a59c700 1 -- 172.21.15.12:0/3658129420 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c06ec70 con 0x7fa80c156a10 2023-07-08T12:39:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa80a59c700 1 --2- 172.21.15.12:0/3658129420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa80c156a10 0x7fa80c06e0c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa80c06c880 tx=0x7fa7f4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1552bfd184982c server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:10.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa80959a700 1 -- 172.21.15.12:0/3658129420 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4005e30 con 0x7fa80c156a10 2023-07-08T12:39:10.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa80959a700 1 -- 172.21.15.12:0/3658129420 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f4006030 con 0x7fa80c156a10 2023-07-08T12:39:10.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.428+0000 7fa80959a700 1 -- 172.21.15.12:0/3658129420 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40063c0 con 0x7fa80c156a10 2023-07-08T12:39:10.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.429+0000 7fa8109f9700 1 -- 172.21.15.12:0/3658129420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa80c156a10 msgr2=0x7fa80c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.429+0000 7fa8109f9700 1 --2- 172.21.15.12:0/3658129420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa80c156a10 0x7fa80c06e0c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa80c06c880 tx=0x7fa7f4009b00 comp rx=0 tx=0).stop 2023-07-08T12:39:10.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.429+0000 7fa8109f9700 1 -- 172.21.15.12:0/3658129420 shutdown_connections 2023-07-08T12:39:10.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.429+0000 7fa8109f9700 1 --2- 172.21.15.12:0/3658129420 >> v2:172.21.15.160:3300/0 conn(0x7fa80c0cc240 0x7fa80c0cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.429+0000 7fa8109f9700 1 --2- 172.21.15.12:0/3658129420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa80c156a10 0x7fa80c06e0c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.429+0000 7fa8109f9700 1 -- 172.21.15.12:0/3658129420 >> 172.21.15.12:0/3658129420 conn(0x7fa80c098b50 msgr2=0x7fa80c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:10.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.429+0000 7fa8109f9700 1 -- 172.21.15.12:0/3658129420 shutdown_connections 2023-07-08T12:39:10.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.429+0000 7fa8109f9700 1 -- 172.21.15.12:0/3658129420 wait complete. 2023-07-08T12:39:10.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.430+0000 7fa8109f9700 1 Processor -- start 2023-07-08T12:39:10.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.430+0000 7fa8109f9700 1 -- start start 2023-07-08T12:39:10.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.431+0000 7fa8109f9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa80c0cc240 0x7fa80c155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.431+0000 7fa8109f9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa80c156a10 0x7fa80c0b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.431+0000 7fa8109f9700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa80c15d390 0x7fa80c0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.431+0000 7fa8109f9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c1f2750 con 0x7fa80c156a10 2023-07-08T12:39:10.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.431+0000 7fa8109f9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c1e7940 con 0x7fa80c0cc240 2023-07-08T12:39:10.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.431+0000 7fa8109f9700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c156170 con 0x7fa80c15d390 2023-07-08T12:39:10.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.431+0000 7fa809d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa80c156a10 0x7fa80c0b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.431+0000 7fa809d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa80c156a10 0x7fa80c0b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51022/0 (socket says 172.21.15.12:51022) 2023-07-08T12:39:10.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.431+0000 7fa809d9b700 1 -- 172.21.15.12:0/1543120509 learned_addr learned my addr 172.21.15.12:0/1543120509 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:10.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.431+0000 7fa80a59c700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa80c0cc240 0x7fa80c155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.431+0000 7fa80ad9d700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa80c15d390 0x7fa80c0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.432+0000 7fa809d9b700 1 -- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa80c15d390 msgr2=0x7fa80c0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.432+0000 7fa809d9b700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa80c15d390 0x7fa80c0b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.432+0000 7fa809d9b700 1 -- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa80c0cc240 msgr2=0x7fa80c155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.432+0000 7fa809d9b700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa80c0cc240 0x7fa80c155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.432+0000 7fa809d9b700 1 -- 172.21.15.12:0/1543120509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f4009430 con 0x7fa80c156a10 2023-07-08T12:39:10.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.432+0000 7fa80ad9d700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa80c15d390 0x7fa80c0b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:10.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.432+0000 7fa809d9b700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa80c156a10 0x7fa80c0b3e80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0011e0 tx=0x7fa7fc00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:10.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.432+0000 7fa802ffd700 1 -- 172.21.15.12:0/1543120509 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc004c80 con 0x7fa80c156a10 2023-07-08T12:39:10.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.432+0000 7fa802ffd700 1 -- 172.21.15.12:0/1543120509 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc004de0 con 0x7fa80c156a10 2023-07-08T12:39:10.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.432+0000 7fa8109f9700 1 -- 172.21.15.12:0/1543120509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80c1f2390 con 0x7fa80c156a10 2023-07-08T12:39:10.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.432+0000 7fa802ffd700 1 -- 172.21.15.12:0/1543120509 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc007310 con 0x7fa80c156a10 2023-07-08T12:39:10.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.432+0000 7fa8109f9700 1 -- 172.21.15.12:0/1543120509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c1e7660 con 0x7fa80c156a10 2023-07-08T12:39:10.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.433+0000 7fa8109f9700 1 -- 172.21.15.12:0/1543120509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa80c1f2890 con 0x7fa80c156a10 2023-07-08T12:39:10.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.433+0000 7fa802ffd700 1 -- 172.21.15.12:0/1543120509 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7fc007470 con 0x7fa80c156a10 2023-07-08T12:39:10.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.434+0000 7fa802ffd700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa7ec06ace0 0x7fa7ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.434+0000 7fa802ffd700 1 -- 172.21.15.12:0/1543120509 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa7fc0894a0 con 0x7fa80c156a10 2023-07-08T12:39:10.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.434+0000 7fa80a59c700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa7ec06ace0 0x7fa7ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.434+0000 7fa80a59c700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa7ec06ace0 0x7fa7ec06d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4000c00 tx=0x7fa7f4008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:10.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.438+0000 7fa802ffd700 1 -- 172.21.15.12:0/1543120509 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa80c1f2890 con 0x7fa80c156a10 2023-07-08T12:39:10.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.636+0000 7fa8109f9700 1 -- 172.21.15.12:0/1543120509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa80c1f2890 con 0x7fa80c156a10 2023-07-08T12:39:10.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.636+0000 7fa802ffd700 1 -- 172.21.15.12:0/1543120509 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa80c1f2890 con 0x7fa80c156a10 2023-07-08T12:39:10.640 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:39:10.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.638+0000 7fa8109f9700 1 -- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa7ec06ace0 msgr2=0x7fa7ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.638+0000 7fa8109f9700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa7ec06ace0 0x7fa7ec06d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4000c00 tx=0x7fa7f4008070 comp rx=0 tx=0).stop 2023-07-08T12:39:10.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.638+0000 7fa8109f9700 1 -- 172.21.15.12:0/1543120509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa80c156a10 msgr2=0x7fa80c0b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.638+0000 7fa8109f9700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa80c156a10 0x7fa80c0b3e80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0011e0 tx=0x7fa7fc00ca30 comp rx=0 tx=0).stop 2023-07-08T12:39:10.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.638+0000 7fa8109f9700 1 -- 172.21.15.12:0/1543120509 shutdown_connections 2023-07-08T12:39:10.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.638+0000 7fa8109f9700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa7ec06ace0 0x7fa7ec06d190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.638+0000 7fa8109f9700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa80c15d390 0x7fa80c0b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.638+0000 7fa8109f9700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa80c0cc240 0x7fa80c155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.638+0000 7fa8109f9700 1 --2- 172.21.15.12:0/1543120509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa80c156a10 0x7fa80c0b3e80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.638+0000 7fa8109f9700 1 -- 172.21.15.12:0/1543120509 >> 172.21.15.12:0/1543120509 conn(0x7fa80c098b50 msgr2=0x7fa80c15c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:10.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.639+0000 7fa8109f9700 1 -- 172.21.15.12:0/1543120509 shutdown_connections 2023-07-08T12:39:10.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.639+0000 7fa8109f9700 1 -- 172.21.15.12:0/1543120509 wait complete. 2023-07-08T12:39:10.653 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-08T12:39:10.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.778+0000 7efcacd44700 1 Processor -- start 2023-07-08T12:39:10.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.778+0000 7efcacd44700 1 -- start start 2023-07-08T12:39:10.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.779+0000 7efcacd44700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efca806ebe0 0x7efca80cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.779+0000 7efcacd44700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca815d770 con 0x7efca80cd9b0 2023-07-08T12:39:10.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.779+0000 7efcacd44700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca80cddc0 con 0x7efca80aeec0 2023-07-08T12:39:10.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.779+0000 7efcacd44700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca806d700 con 0x7efca806ebe0 2023-07-08T12:39:10.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.779+0000 7efca6d9d700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7efca80cd9b0 0x7efca80cbe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38426/0 (socket says 172.21.15.12:38426) 2023-07-08T12:39:10.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.779+0000 7efca6d9d700 1 -- 172.21.15.12:0/784729078 learned_addr learned my addr 172.21.15.12:0/784729078 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:10.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.780+0000 7efca5d9b700 1 --2- 172.21.15.12:0/784729078 >> v2:172.21.15.160:3300/0 conn(0x7efca806ebe0 0x7efca80cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.780+0000 7efca559a700 1 -- 172.21.15.12:0/784729078 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714586456 0 0) 0x7efca815d770 con 0x7efca80cd9b0 2023-07-08T12:39:10.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.780+0000 7efca559a700 1 -- 172.21.15.12:0/784729078 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc900035b0 con 0x7efca80cd9b0 2023-07-08T12:39:10.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.780+0000 7efca559a700 1 -- 172.21.15.12:0/784729078 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203992561 0 0) 0x7efca80cddc0 con 0x7efca80aeec0 2023-07-08T12:39:10.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.780+0000 7efca559a700 1 -- 172.21.15.12:0/784729078 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efca815d770 con 0x7efca80aeec0 2023-07-08T12:39:10.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.780+0000 7efca559a700 1 -- 172.21.15.12:0/784729078 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 392159703 0 0) 0x7efc900035b0 con 0x7efca80cd9b0 2023-07-08T12:39:10.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.780+0000 7efca5d9b700 1 -- 172.21.15.12:0/784729078 >> v1:172.21.15.160:6789/0 conn(0x7efca80aeec0 legacy=0x7efca806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.780+0000 7efca5d9b700 1 -- 172.21.15.12:0/784729078 >> v1:172.21.15.12:6789/0 conn(0x7efca80cd9b0 legacy=0x7efca80cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.780+0000 7efca5d9b700 1 -- 172.21.15.12:0/784729078 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca815cb60 con 0x7efca806ebe0 2023-07-08T12:39:10.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.780+0000 7efca5d9b700 1 --2- 172.21.15.12:0/784729078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efca806ebe0 0x7efca80cd470 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efc98009840 tx=0x7efc98009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e1bee915f29c7df server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:10.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.780+0000 7efca559a700 1 -- 172.21.15.12:0/784729078 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98011830 con 0x7efca806ebe0 2023-07-08T12:39:10.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.780+0000 7efca559a700 1 -- 172.21.15.12:0/784729078 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc98011990 con 0x7efca806ebe0 2023-07-08T12:39:10.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.780+0000 7efca559a700 1 -- 172.21.15.12:0/784729078 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98011d20 con 0x7efca806ebe0 2023-07-08T12:39:10.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.781+0000 7efcacd44700 1 -- 172.21.15.12:0/784729078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efca806ebe0 msgr2=0x7efca80cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.781+0000 7efcacd44700 1 --2- 172.21.15.12:0/784729078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efca806ebe0 0x7efca80cd470 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efc98009840 tx=0x7efc98009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:10.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.781+0000 7efcacd44700 1 -- 172.21.15.12:0/784729078 shutdown_connections 2023-07-08T12:39:10.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.781+0000 7efcacd44700 1 --2- 172.21.15.12:0/784729078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efca806ebe0 0x7efca80cd470 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.781+0000 7efcacd44700 1 -- 172.21.15.12:0/784729078 >> 172.21.15.12:0/784729078 conn(0x7efca8098b50 msgr2=0x7efca806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:10.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.781+0000 7efcacd44700 1 -- 172.21.15.12:0/784729078 shutdown_connections 2023-07-08T12:39:10.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.781+0000 7efcacd44700 1 -- 172.21.15.12:0/784729078 wait complete. 2023-07-08T12:39:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.782+0000 7efcacd44700 1 Processor -- start 2023-07-08T12:39:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.782+0000 7efcacd44700 1 -- start start 2023-07-08T12:39:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.783+0000 7efcacd44700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efca806ebe0 0x7efca80af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.783+0000 7efcacd44700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca80aeec0 0x7efca80b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.783+0000 7efcacd44700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efca80cd9b0 0x7efca8149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.783+0000 7efcacd44700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca80b2660 con 0x7efca80aeec0 2023-07-08T12:39:10.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.783+0000 7efcacd44700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca80c35a0 con 0x7efca80cd9b0 2023-07-08T12:39:10.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.783+0000 7efcacd44700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca80c3880 con 0x7efca806ebe0 2023-07-08T12:39:10.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.783+0000 7efca5d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca80aeec0 0x7efca80b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.783+0000 7efca5d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca80aeec0 0x7efca80b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51036/0 (socket says 172.21.15.12:51036) 2023-07-08T12:39:10.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.783+0000 7efca5d9b700 1 -- 172.21.15.12:0/133994478 learned_addr learned my addr 172.21.15.12:0/133994478 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:10.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.783+0000 7efca6d9d700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efca80cd9b0 0x7efca8149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.784+0000 7efca659c700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efca806ebe0 0x7efca80af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.784+0000 7efca5d9b700 1 -- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efca806ebe0 msgr2=0x7efca80af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.784+0000 7efca5d9b700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efca806ebe0 0x7efca80af460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.784+0000 7efca5d9b700 1 -- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efca80cd9b0 msgr2=0x7efca8149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.784+0000 7efca5d9b700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efca80cd9b0 0x7efca8149e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.784+0000 7efca5d9b700 1 -- 172.21.15.12:0/133994478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc980094a0 con 0x7efca80aeec0 2023-07-08T12:39:10.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.784+0000 7efca6d9d700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efca80cd9b0 0x7efca8149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:10.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.784+0000 7efca5d9b700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca80aeec0 0x7efca80b1820 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efc98014040 tx=0x7efc980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.784+0000 7efc96ffd700 1 -- 172.21.15.12:0/133994478 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9801ba10 con 0x7efca80aeec0 2023-07-08T12:39:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.784+0000 7efc96ffd700 1 -- 172.21.15.12:0/133994478 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9801bb70 con 0x7efca80aeec0 2023-07-08T12:39:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.784+0000 7efcacd44700 1 -- 172.21.15.12:0/133994478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca80c3b60 con 0x7efca80aeec0 2023-07-08T12:39:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.784+0000 7efc96ffd700 1 -- 172.21.15.12:0/133994478 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc980253f0 con 0x7efca80aeec0 2023-07-08T12:39:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.784+0000 7efcacd44700 1 -- 172.21.15.12:0/133994478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca80b1d60 con 0x7efca80aeec0 2023-07-08T12:39:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.785+0000 7efcacd44700 1 -- 172.21.15.12:0/133994478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca80c4080 con 0x7efca80aeec0 2023-07-08T12:39:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.785+0000 7efc96ffd700 1 -- 172.21.15.12:0/133994478 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc9802e3b0 con 0x7efca80aeec0 2023-07-08T12:39:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.786+0000 7efc96ffd700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efc8806ace0 0x7efc8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.786+0000 7efc96ffd700 1 -- 172.21.15.12:0/133994478 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efc9808ca80 con 0x7efca80aeec0 2023-07-08T12:39:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.786+0000 7efca659c700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efc8806ace0 0x7efc8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.787+0000 7efca659c700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efc8806ace0 0x7efc8806d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efc9c001230 tx=0x7efc9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:10.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.790+0000 7efc96ffd700 1 -- 172.21.15.12:0/133994478 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efca80c4080 con 0x7efca80aeec0 2023-07-08T12:39:10.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.989+0000 7efcacd44700 1 -- 172.21.15.12:0/133994478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efca80c4080 con 0x7efca80aeec0 2023-07-08T12:39:10.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.989+0000 7efc96ffd700 1 -- 172.21.15.12:0/133994478 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7efc98015020 con 0x7efca80aeec0 2023-07-08T12:39:10.992 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:10.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.991+0000 7efcacd44700 1 -- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efc8806ace0 msgr2=0x7efc8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.991+0000 7efcacd44700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efc8806ace0 0x7efc8806d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efc9c001230 tx=0x7efc9c009040 comp rx=0 tx=0).stop 2023-07-08T12:39:10.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.991+0000 7efcacd44700 1 -- 172.21.15.12:0/133994478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca80aeec0 msgr2=0x7efca80b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:10.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.991+0000 7efcacd44700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca80aeec0 0x7efca80b1820 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efc98014040 tx=0x7efc980047c0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.992+0000 7efcacd44700 1 -- 172.21.15.12:0/133994478 shutdown_connections 2023-07-08T12:39:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.992+0000 7efcacd44700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7efc8806ace0 0x7efc8806d190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.992+0000 7efcacd44700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efca80cd9b0 0x7efca8149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.992+0000 7efcacd44700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efca806ebe0 0x7efca80af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.992+0000 7efcacd44700 1 --2- 172.21.15.12:0/133994478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca80aeec0 0x7efca80b1820 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.992+0000 7efcacd44700 1 -- 172.21.15.12:0/133994478 >> 172.21.15.12:0/133994478 conn(0x7efca8098b50 msgr2=0x7efca8154d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.992+0000 7efcacd44700 1 -- 172.21.15.12:0/133994478 shutdown_connections 2023-07-08T12:39:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:10.992+0000 7efcacd44700 1 -- 172.21.15.12:0/133994478 wait complete. 2023-07-08T12:39:11.006 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-08T12:39:11.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.131+0000 7f977e920700 1 Processor -- start 2023-07-08T12:39:11.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.131+0000 7f977e920700 1 -- start start 2023-07-08T12:39:11.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.132+0000 7f977e920700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9778156a10 0x7f977806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.132+0000 7f977e920700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f977815d390 0x7f97781608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.132+0000 7f977e920700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f97780cc240 0x7f97780cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.132+0000 7f977e920700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97780ccc00 con 0x7f977815d390 2023-07-08T12:39:11.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.132+0000 7f977e920700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9778160410 con 0x7f97780cc240 2023-07-08T12:39:11.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.132+0000 7f977e920700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9778156de0 con 0x7f9778156a10 2023-07-08T12:39:11.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.132+0000 7f97777fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f977815d390 0x7f97781608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.132+0000 7f97777fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f977815d390 0x7f97781608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51042/0 (socket says 172.21.15.12:51042) 2023-07-08T12:39:11.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.132+0000 7f97777fe700 1 -- 172.21.15.12:0/785424660 learned_addr learned my addr 172.21.15.12:0/785424660 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:11.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.132+0000 7f977cebd700 1 --2- 172.21.15.12:0/785424660 >> v2:172.21.15.160:3300/0 conn(0x7f97780cc240 0x7f97780cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.132+0000 7f9777fff700 1 --2- 172.21.15.12:0/785424660 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9778156a10 0x7f977806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.133+0000 7f97777fe700 1 -- 172.21.15.12:0/785424660 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9778156a10 msgr2=0x7f977806e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:11.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.133+0000 7f97777fe700 1 --2- 172.21.15.12:0/785424660 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9778156a10 0x7f977806e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.133+0000 7f97777fe700 1 -- 172.21.15.12:0/785424660 >> v2:172.21.15.160:3300/0 conn(0x7f97780cc240 msgr2=0x7f97780cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:11.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.133+0000 7f97777fe700 1 --2- 172.21.15.12:0/785424660 >> v2:172.21.15.160:3300/0 conn(0x7f97780cc240 0x7f97780cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.133+0000 7f97777fe700 1 -- 172.21.15.12:0/785424660 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977806ec70 con 0x7f977815d390 2023-07-08T12:39:11.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.133+0000 7f97777fe700 1 --2- 172.21.15.12:0/785424660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977815d390 0x7f97781608a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f97780b19e0 tx=0x7f9760009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd725ca588a8d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:11.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.133+0000 7f9776ffd700 1 -- 172.21.15.12:0/785424660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9760011830 con 0x7f977815d390 2023-07-08T12:39:11.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.133+0000 7f9776ffd700 1 -- 172.21.15.12:0/785424660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9760011990 con 0x7f977815d390 2023-07-08T12:39:11.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.133+0000 7f9776ffd700 1 -- 172.21.15.12:0/785424660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9760011d20 con 0x7f977815d390 2023-07-08T12:39:11.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.133+0000 7f977e920700 1 -- 172.21.15.12:0/785424660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977815d390 msgr2=0x7f97781608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.133+0000 7f977e920700 1 --2- 172.21.15.12:0/785424660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977815d390 0x7f97781608a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f97780b19e0 tx=0x7f9760009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.133+0000 7f977e920700 1 -- 172.21.15.12:0/785424660 shutdown_connections 2023-07-08T12:39:11.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.134+0000 7f977e920700 1 --2- 172.21.15.12:0/785424660 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9778156a10 0x7f977806e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.134+0000 7f977e920700 1 --2- 172.21.15.12:0/785424660 >> v2:172.21.15.160:3300/0 conn(0x7f97780cc240 0x7f97780cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.134+0000 7f977e920700 1 --2- 172.21.15.12:0/785424660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977815d390 0x7f97781608a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.134+0000 7f977e920700 1 -- 172.21.15.12:0/785424660 >> 172.21.15.12:0/785424660 conn(0x7f9778098b50 msgr2=0x7f977806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:11.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.134+0000 7f977e920700 1 -- 172.21.15.12:0/785424660 shutdown_connections 2023-07-08T12:39:11.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.134+0000 7f977e920700 1 -- 172.21.15.12:0/785424660 wait complete. 2023-07-08T12:39:11.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.135+0000 7f977e920700 1 Processor -- start 2023-07-08T12:39:11.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.135+0000 7f977e920700 1 -- start start 2023-07-08T12:39:11.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977e920700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97780cc240 0x7f9778159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977e920700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9778156a10 0x7f97781f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977e920700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977815d390 0x7f97780b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977e920700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97780b5670 con 0x7f977815d390 2023-07-08T12:39:11.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977e920700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97780b50b0 con 0x7f97780cc240 2023-07-08T12:39:11.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977e920700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97780b5390 con 0x7f9778156a10 2023-07-08T12:39:11.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977cebd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977815d390 0x7f97780b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977cebd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977815d390 0x7f97780b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51052/0 (socket says 172.21.15.12:51052) 2023-07-08T12:39:11.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977cebd700 1 -- 172.21.15.12:0/1552998345 learned_addr learned my addr 172.21.15.12:0/1552998345 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:11.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f9777fff700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97780cc240 0x7f9778159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977cebd700 1 -- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9778156a10 msgr2=0x7f97781f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f97777fe700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9778156a10 0x7f97781f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977cebd700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9778156a10 0x7f97781f6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977cebd700 1 -- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97780cc240 msgr2=0x7f9778159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977cebd700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97780cc240 0x7f9778159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977cebd700 1 -- 172.21.15.12:0/1552998345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9760009550 con 0x7f977815d390 2023-07-08T12:39:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f9777fff700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97780cc240 0x7f9778159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977cebd700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977815d390 0x7f97780b4810 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f977000eed0 tx=0x7f977000b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f9774ff9700 1 -- 172.21.15.12:0/1552998345 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770010040 con 0x7f977815d390 2023-07-08T12:39:11.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f9774ff9700 1 -- 172.21.15.12:0/1552998345 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977000bb70 con 0x7f977815d390 2023-07-08T12:39:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977e920700 1 -- 172.21.15.12:0/1552998345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97781f5800 con 0x7f977815d390 2023-07-08T12:39:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f9774ff9700 1 -- 172.21.15.12:0/1552998345 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770014400 con 0x7f977815d390 2023-07-08T12:39:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.136+0000 7f977e920700 1 -- 172.21.15.12:0/1552998345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97781f6540 con 0x7f977815d390 2023-07-08T12:39:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.137+0000 7f977e920700 1 -- 172.21.15.12:0/1552998345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97781f5d20 con 0x7f977815d390 2023-07-08T12:39:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.138+0000 7f9774ff9700 1 -- 172.21.15.12:0/1552998345 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9770014560 con 0x7f977815d390 2023-07-08T12:39:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.138+0000 7f9774ff9700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f975806ace0 0x7f975806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.138+0000 7f9774ff9700 1 -- 172.21.15.12:0/1552998345 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9770089b60 con 0x7f977815d390 2023-07-08T12:39:11.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.139+0000 7f9777fff700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f975806ace0 0x7f975806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.139+0000 7f9777fff700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f975806ace0 0x7f975806d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9768009b40 tx=0x7f9768008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:11.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.143+0000 7f9774ff9700 1 -- 172.21.15.12:0/1552998345 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9770054970 con 0x7f977815d390 2023-07-08T12:39:11.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.339+0000 7f977e920700 1 -- 172.21.15.12:0/1552998345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97780cd6b0 con 0x7f977815d390 2023-07-08T12:39:11.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.340+0000 7f9774ff9700 1 -- 172.21.15.12:0/1552998345 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9770018020 con 0x7f977815d390 2023-07-08T12:39:11.342 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-07-08T12:39:11.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.341+0000 7f977e920700 1 -- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f975806ace0 msgr2=0x7f975806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.341+0000 7f977e920700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f975806ace0 0x7f975806d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9768009b40 tx=0x7f9768008040 comp rx=0 tx=0).stop 2023-07-08T12:39:11.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.342+0000 7f977e920700 1 -- 172.21.15.12:0/1552998345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977815d390 msgr2=0x7f97780b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.342+0000 7f977e920700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977815d390 0x7f97780b4810 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f977000eed0 tx=0x7f977000b0b0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.342+0000 7f977e920700 1 -- 172.21.15.12:0/1552998345 shutdown_connections 2023-07-08T12:39:11.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.342+0000 7f977e920700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f975806ace0 0x7f975806d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.342+0000 7f977e920700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977815d390 0x7f97780b4810 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.342+0000 7f977e920700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97780cc240 0x7f9778159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.342+0000 7f977e920700 1 --2- 172.21.15.12:0/1552998345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9778156a10 0x7f97781f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.342+0000 7f977e920700 1 -- 172.21.15.12:0/1552998345 >> 172.21.15.12:0/1552998345 conn(0x7f9778098b50 msgr2=0x7f977815f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:11.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.342+0000 7f977e920700 1 -- 172.21.15.12:0/1552998345 shutdown_connections 2023-07-08T12:39:11.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.342+0000 7f977e920700 1 -- 172.21.15.12:0/1552998345 wait complete. 2023-07-08T12:39:11.356 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-08T12:39:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.481+0000 7fac646bd700 1 Processor -- start 2023-07-08T12:39:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.481+0000 7fac646bd700 1 -- start start 2023-07-08T12:39:11.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.482+0000 7fac646bd700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fac5c15a6d0 0x7fac5c15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.482+0000 7fac646bd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c161ce0 0x7fac5c1640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.482+0000 7fac646bd700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5c158130 con 0x7fac5c0ce560 2023-07-08T12:39:11.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.482+0000 7fac646bd700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac5c15b730 con 0x7fac5c15a6d0 2023-07-08T12:39:11.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.482+0000 7fac646bd700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c15b450 con 0x7fac5c161ce0 2023-07-08T12:39:11.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac62459700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fac5c0ce560 0x7fac5c0ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38452/0 (socket says 172.21.15.12:38452) 2023-07-08T12:39:11.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac62459700 1 -- 172.21.15.12:0/1386069434 learned_addr learned my addr 172.21.15.12:0/1386069434 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:11.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61c58700 1 --2- 172.21.15.12:0/1386069434 >> v2:172.21.15.160:3300/0 conn(0x7fac5c15a6d0 0x7fac5c15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac62c5a700 1 --2- 172.21.15.12:0/1386069434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c161ce0 0x7fac5c1640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643033827 0 0) 0x7fac5c158130 con 0x7fac5c0ce560 2023-07-08T12:39:11.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac480035b0 con 0x7fac5c0ce560 2023-07-08T12:39:11.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3261811964 0 0) 0x7fac480035b0 con 0x7fac5c0ce560 2023-07-08T12:39:11.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c161ce0 msgr2=0x7fac5c1640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:11.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 --2- 172.21.15.12:0/1386069434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c161ce0 0x7fac5c1640c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 >> v2:172.21.15.160:3300/0 conn(0x7fac5c15a6d0 msgr2=0x7fac5c15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:11.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 --2- 172.21.15.12:0/1386069434 >> v2:172.21.15.160:3300/0 conn(0x7fac5c15a6d0 0x7fac5c15ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5c06e850 con 0x7fac5c0ce560 2023-07-08T12:39:11.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7fac58001670 con 0x7fac5c0ce560 2023-07-08T12:39:11.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 >> v1:172.21.15.12:6789/0 conn(0x7fac5c0ce560 legacy=0x7fac5c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 --2- 172.21.15.12:0/1386069434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c161ce0 0x7fac48003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 --2- 172.21.15.12:0/1386069434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac48003f80 0x7fac48004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 --2- 172.21.15.12:0/1386069434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac48004d90 0x7fac48005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c15b730 con 0x7fac48003f80 2023-07-08T12:39:11.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c15b450 con 0x7fac48004d90 2023-07-08T12:39:11.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4800ba10 con 0x7fac5c161ce0 2023-07-08T12:39:11.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac61c58700 1 --2- 172.21.15.12:0/1386069434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac48003f80 0x7fac48004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.483+0000 7fac62459700 1 --2- 172.21.15.12:0/1386069434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c161ce0 0x7fac48003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.484+0000 7fac62c5a700 1 --2- 172.21.15.12:0/1386069434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac48004d90 0x7fac48005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.484+0000 7fac61c58700 1 -- 172.21.15.12:0/1386069434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c161ce0 msgr2=0x7fac48003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:11.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.484+0000 7fac61c58700 1 --2- 172.21.15.12:0/1386069434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c161ce0 0x7fac48003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.484+0000 7fac61c58700 1 -- 172.21.15.12:0/1386069434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac48004d90 msgr2=0x7fac48005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:11.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.484+0000 7fac61c58700 1 --2- 172.21.15.12:0/1386069434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac48004d90 0x7fac48005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.484+0000 7fac61c58700 1 -- 172.21.15.12:0/1386069434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac480035b0 con 0x7fac48003f80 2023-07-08T12:39:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.484+0000 7fac61c58700 1 --2- 172.21.15.12:0/1386069434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac48003f80 0x7fac48004850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fac50009b60 tx=0x7fac48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1fe9da01e7e94e server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.484+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5000e6c0 con 0x7fac48003f80 2023-07-08T12:39:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.484+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5000e820 con 0x7fac48003f80 2023-07-08T12:39:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.484+0000 7fac61457700 1 -- 172.21.15.12:0/1386069434 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5000eb90 con 0x7fac48003f80 2023-07-08T12:39:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.485+0000 7fac646bd700 1 -- 172.21.15.12:0/1386069434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac48003f80 msgr2=0x7fac48004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.485+0000 7fac646bd700 1 --2- 172.21.15.12:0/1386069434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac48003f80 0x7fac48004850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fac50009b60 tx=0x7fac48005880 comp rx=0 tx=0).stop 2023-07-08T12:39:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.485+0000 7fac62c5a700 1 -- 172.21.15.12:0/1386069434 reap_dead start 2023-07-08T12:39:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.485+0000 7fac646bd700 1 -- 172.21.15.12:0/1386069434 shutdown_connections 2023-07-08T12:39:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.485+0000 7fac646bd700 1 -- 172.21.15.12:0/1386069434 >> 172.21.15.12:0/1386069434 conn(0x7fac5c099c60 msgr2=0x7fac5c089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.485+0000 7fac646bd700 1 -- 172.21.15.12:0/1386069434 shutdown_connections 2023-07-08T12:39:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.485+0000 7fac646bd700 1 -- 172.21.15.12:0/1386069434 wait complete. 2023-07-08T12:39:11.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.486+0000 7fac646bd700 1 Processor -- start 2023-07-08T12:39:11.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.486+0000 7fac646bd700 1 -- start start 2023-07-08T12:39:11.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac646bd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac48003f80 0x7fac5c14de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac646bd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c0af2c0 0x7fac5c0ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac646bd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac5c14f7a0 0x7fac5c14fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac646bd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c0aea20 con 0x7fac5c14f7a0 2023-07-08T12:39:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac646bd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c150150 con 0x7fac48003f80 2023-07-08T12:39:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac646bd700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c14e340 con 0x7fac5c0af2c0 2023-07-08T12:39:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac62c5a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac5c14f7a0 0x7fac5c14fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac62c5a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac5c14f7a0 0x7fac5c14fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51072/0 (socket says 172.21.15.12:51072) 2023-07-08T12:39:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac62c5a700 1 -- 172.21.15.12:0/477000051 learned_addr learned my addr 172.21.15.12:0/477000051 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:11.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac62459700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac48003f80 0x7fac5c14de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac61c58700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c0af2c0 0x7fac5c0ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac62c5a700 1 -- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c0af2c0 msgr2=0x7fac5c0ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac62c5a700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c0af2c0 0x7fac5c0ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac62c5a700 1 -- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac48003f80 msgr2=0x7fac5c14de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac62c5a700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac48003f80 0x7fac5c14de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac62c5a700 1 -- 172.21.15.12:0/477000051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5c0adda0 con 0x7fac5c14f7a0 2023-07-08T12:39:11.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac61c58700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c0af2c0 0x7fac5c0ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:11.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac62c5a700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac5c14f7a0 0x7fac5c14fc10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fac5c08a780 tx=0x7fac54011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:11.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac4effd700 1 -- 172.21.15.12:0/477000051 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5400e040 con 0x7fac5c14f7a0 2023-07-08T12:39:11.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac4effd700 1 -- 172.21.15.12:0/477000051 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5400f040 con 0x7fac5c14f7a0 2023-07-08T12:39:11.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac646bd700 1 -- 172.21.15.12:0/477000051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5c0ae080 con 0x7fac5c14f7a0 2023-07-08T12:39:11.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.487+0000 7fac4effd700 1 -- 172.21.15.12:0/477000051 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5400fa00 con 0x7fac5c14f7a0 2023-07-08T12:39:11.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.488+0000 7fac646bd700 1 -- 172.21.15.12:0/477000051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5c0ae640 con 0x7fac5c14f7a0 2023-07-08T12:39:11.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.488+0000 7fac646bd700 1 -- 172.21.15.12:0/477000051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac5c0ae7d0 con 0x7fac5c14f7a0 2023-07-08T12:39:11.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.489+0000 7fac4effd700 1 -- 172.21.15.12:0/477000051 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fac5400fb60 con 0x7fac5c14f7a0 2023-07-08T12:39:11.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.490+0000 7fac4effd700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac4006ace0 0x7fac4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.490+0000 7fac4effd700 1 -- 172.21.15.12:0/477000051 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fac5408aa60 con 0x7fac5c14f7a0 2023-07-08T12:39:11.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.490+0000 7fac62459700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac4006ace0 0x7fac4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.490+0000 7fac62459700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac4006ace0 0x7fac4006d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac58000c00 tx=0x7fac58009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:11.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.493+0000 7fac4effd700 1 -- 172.21.15.12:0/477000051 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fac540557c0 con 0x7fac5c14f7a0 2023-07-08T12:39:11.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.693+0000 7fac646bd700 1 -- 172.21.15.12:0/477000051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fac5c0cf2f0 con 0x7fac5c14f7a0 2023-07-08T12:39:11.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.694+0000 7fac4effd700 1 -- 172.21.15.12:0/477000051 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fac54058de0 con 0x7fac5c14f7a0 2023-07-08T12:39:11.697 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:11.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.696+0000 7fac646bd700 1 -- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac4006ace0 msgr2=0x7fac4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.696+0000 7fac646bd700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac4006ace0 0x7fac4006d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac58000c00 tx=0x7fac58009040 comp rx=0 tx=0).stop 2023-07-08T12:39:11.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.696+0000 7fac646bd700 1 -- 172.21.15.12:0/477000051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac5c14f7a0 msgr2=0x7fac5c14fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.696+0000 7fac646bd700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac5c14f7a0 0x7fac5c14fc10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fac5c08a780 tx=0x7fac54011f80 comp rx=0 tx=0).stop 2023-07-08T12:39:11.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.696+0000 7fac646bd700 1 -- 172.21.15.12:0/477000051 shutdown_connections 2023-07-08T12:39:11.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.696+0000 7fac646bd700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac4006ace0 0x7fac4006d190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.696+0000 7fac646bd700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac5c14f7a0 0x7fac5c14fc10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.696+0000 7fac646bd700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac48003f80 0x7fac5c14de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.696+0000 7fac646bd700 1 --2- 172.21.15.12:0/477000051 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac5c0af2c0 0x7fac5c0ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.696+0000 7fac646bd700 1 -- 172.21.15.12:0/477000051 >> 172.21.15.12:0/477000051 conn(0x7fac5c099c60 msgr2=0x7fac5c072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:11.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.696+0000 7fac646bd700 1 -- 172.21.15.12:0/477000051 shutdown_connections 2023-07-08T12:39:11.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.696+0000 7fac646bd700 1 -- 172.21.15.12:0/477000051 wait complete. 2023-07-08T12:39:11.711 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-08T12:39:11.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.834+0000 7f29f98f4700 1 Processor -- start 2023-07-08T12:39:11.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.834+0000 7f29f98f4700 1 -- start start 2023-07-08T12:39:11.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.835+0000 7f29f98f4700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f29f415c460 0x7f29f40b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.835+0000 7f29f98f4700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29f40d2640 0x7f29f40cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.835+0000 7f29f98f4700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f40d1790 con 0x7f29f40cf720 2023-07-08T12:39:11.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.835+0000 7f29f98f4700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f40cedd0 con 0x7f29f415c460 2023-07-08T12:39:11.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.835+0000 7f29f98f4700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f40cd950 con 0x7f29f40d2640 2023-07-08T12:39:11.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.835+0000 7f29f27fc700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29f40d2640 0x7f29f40cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.835+0000 7f29f2ffd700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f29f415c460 0x7f29f40b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.835+0000 7f29f37fe700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f29f40cf720 0x7f29f40d2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:47698/0 (socket says 172.21.15.12:47698) 2023-07-08T12:39:11.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.835+0000 7f29f37fe700 1 -- 172.21.15.12:0/4124765579 learned_addr learned my addr 172.21.15.12:0/4124765579 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:11.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.836+0000 7f29f1ffb700 1 -- 172.21.15.12:0/4124765579 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682936126 0 0) 0x7f29f40d1790 con 0x7f29f40cf720 2023-07-08T12:39:11.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.836+0000 7f29f1ffb700 1 -- 172.21.15.12:0/4124765579 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29dc0035b0 con 0x7f29f40cf720 2023-07-08T12:39:11.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.836+0000 7f29f2ffd700 1 -- 172.21.15.12:0/4124765579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29f40d2640 msgr2=0x7f29f40cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:11.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.836+0000 7f29f2ffd700 1 --2- 172.21.15.12:0/4124765579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29f40d2640 0x7f29f40cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.836+0000 7f29f2ffd700 1 -- 172.21.15.12:0/4124765579 >> v1:172.21.15.160:6789/0 conn(0x7f29f40cf720 legacy=0x7f29f40d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.836+0000 7f29f2ffd700 1 -- 172.21.15.12:0/4124765579 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f40d14b0 con 0x7f29f415c460 2023-07-08T12:39:11.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.836+0000 7f29f2ffd700 1 --2- 172.21.15.12:0/4124765579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29f415c460 0x7f29f40b59d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009840 tx=0x7f29e8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1025c0717c640786 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:11.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.836+0000 7f29f1ffb700 1 -- 172.21.15.12:0/4124765579 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8011830 con 0x7f29f415c460 2023-07-08T12:39:11.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.836+0000 7f29f1ffb700 1 -- 172.21.15.12:0/4124765579 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e8011990 con 0x7f29f415c460 2023-07-08T12:39:11.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.836+0000 7f29f1ffb700 1 -- 172.21.15.12:0/4124765579 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8011d20 con 0x7f29f415c460 2023-07-08T12:39:11.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.837+0000 7f29f98f4700 1 -- 172.21.15.12:0/4124765579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29f415c460 msgr2=0x7f29f40b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.837+0000 7f29f98f4700 1 --2- 172.21.15.12:0/4124765579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29f415c460 0x7f29f40b59d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009840 tx=0x7f29e8009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:11.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.837+0000 7f29f98f4700 1 -- 172.21.15.12:0/4124765579 shutdown_connections 2023-07-08T12:39:11.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.837+0000 7f29f98f4700 1 --2- 172.21.15.12:0/4124765579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29f415c460 0x7f29f40b59d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.837+0000 7f29f98f4700 1 --2- 172.21.15.12:0/4124765579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29f40d2640 0x7f29f40cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.837+0000 7f29f98f4700 1 -- 172.21.15.12:0/4124765579 >> 172.21.15.12:0/4124765579 conn(0x7f29f4071660 msgr2=0x7f29f4071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:11.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.837+0000 7f29f98f4700 1 -- 172.21.15.12:0/4124765579 shutdown_connections 2023-07-08T12:39:11.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.837+0000 7f29f98f4700 1 -- 172.21.15.12:0/4124765579 wait complete. 2023-07-08T12:39:11.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.838+0000 7f29f98f4700 1 Processor -- start 2023-07-08T12:39:11.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.838+0000 7f29f98f4700 1 -- start start 2023-07-08T12:39:11.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f98f4700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29f40cf720 0x7f29f40c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f98f4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29f40d2640 0x7f29f415b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f98f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29f415c460 0x7f29f40b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f98f4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f40c7460 con 0x7f29f415c460 2023-07-08T12:39:11.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f98f4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f40c67e0 con 0x7f29f40d2640 2023-07-08T12:39:11.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f98f4700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f40c6ac0 con 0x7f29f40cf720 2023-07-08T12:39:11.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29f415c460 0x7f29f40b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29f415c460 0x7f29f40b5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51086/0 (socket says 172.21.15.12:51086) 2023-07-08T12:39:11.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f37fe700 1 -- 172.21.15.12:0/498246092 learned_addr learned my addr 172.21.15.12:0/498246092 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:11.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f27fc700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29f40d2640 0x7f29f415b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f2ffd700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29f40cf720 0x7f29f40c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f37fe700 1 -- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29f40cf720 msgr2=0x7f29f40c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f37fe700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29f40cf720 0x7f29f40c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f37fe700 1 -- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29f40d2640 msgr2=0x7f29f415b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:11.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f37fe700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29f40d2640 0x7f29f415b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:11.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.839+0000 7f29f37fe700 1 -- 172.21.15.12:0/498246092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80094a0 con 0x7f29f415c460 2023-07-08T12:39:11.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.840+0000 7f29f37fe700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29f415c460 0x7f29f40b5cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00caa0 tx=0x7f29ec00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:11.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.840+0000 7f29dbfff700 1 -- 172.21.15.12:0/498246092 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec00a9e0 con 0x7f29f415c460 2023-07-08T12:39:11.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.840+0000 7f29dbfff700 1 -- 172.21.15.12:0/498246092 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ec00ab40 con 0x7f29f415c460 2023-07-08T12:39:11.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.840+0000 7f29f98f4700 1 -- 172.21.15.12:0/498246092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f40c6da0 con 0x7f29f415c460 2023-07-08T12:39:11.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.840+0000 7f29dbfff700 1 -- 172.21.15.12:0/498246092 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec012400 con 0x7f29f415c460 2023-07-08T12:39:11.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.840+0000 7f29f98f4700 1 -- 172.21.15.12:0/498246092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f40c7dc0 con 0x7f29f415c460 2023-07-08T12:39:11.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.841+0000 7f29f98f4700 1 -- 172.21.15.12:0/498246092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f40c7fe0 con 0x7f29f415c460 2023-07-08T12:39:11.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.841+0000 7f29dbfff700 1 -- 172.21.15.12:0/498246092 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f29ec01b3b0 con 0x7f29f415c460 2023-07-08T12:39:11.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.842+0000 7f29dbfff700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f29e006ace0 0x7f29e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:11.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.842+0000 7f29dbfff700 1 -- 172.21.15.12:0/498246092 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f29ec0893c0 con 0x7f29f415c460 2023-07-08T12:39:11.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.842+0000 7f29f2ffd700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f29e006ace0 0x7f29e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:11.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.842+0000 7f29f2ffd700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f29e006ace0 0x7f29e006d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009fd0 tx=0x7f29e8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:11.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:11.846+0000 7f29dbfff700 1 -- 172.21.15.12:0/498246092 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f29f40c7fe0 con 0x7f29f415c460 2023-07-08T12:39:12.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.060+0000 7f29f98f4700 1 -- 172.21.15.12:0/498246092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f29f40c75a0 con 0x7f29f415c460 2023-07-08T12:39:12.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.060+0000 7f29dbfff700 1 -- 172.21.15.12:0/498246092 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f29f40c75a0 con 0x7f29f415c460 2023-07-08T12:39:12.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.062+0000 7f29f98f4700 1 -- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f29e006ace0 msgr2=0x7f29e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.062+0000 7f29f98f4700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f29e006ace0 0x7f29e006d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009fd0 tx=0x7f29e8008030 comp rx=0 tx=0).stop 2023-07-08T12:39:12.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.062+0000 7f29f98f4700 1 -- 172.21.15.12:0/498246092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29f415c460 msgr2=0x7f29f40b5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.062+0000 7f29f98f4700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29f415c460 0x7f29f40b5cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00caa0 tx=0x7f29ec00cdb0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.062+0000 7f29f98f4700 1 -- 172.21.15.12:0/498246092 shutdown_connections 2023-07-08T12:39:12.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.062+0000 7f29f98f4700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f29e006ace0 0x7f29e006d190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.062+0000 7f29f98f4700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29f415c460 0x7f29f40b5cc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.062+0000 7f29f98f4700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29f40cf720 0x7f29f40c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.062+0000 7f29f98f4700 1 --2- 172.21.15.12:0/498246092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29f40d2640 0x7f29f415b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.062+0000 7f29f98f4700 1 -- 172.21.15.12:0/498246092 >> 172.21.15.12:0/498246092 conn(0x7f29f4071660 msgr2=0x7f29f4154d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:12.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.062+0000 7f29f98f4700 1 -- 172.21.15.12:0/498246092 shutdown_connections 2023-07-08T12:39:12.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.062+0000 7f29f98f4700 1 -- 172.21.15.12:0/498246092 wait complete. 2023-07-08T12:39:12.066 INFO:teuthology.orchestra.run.smithi012.stderr:listed 0 entries 2023-07-08T12:39:12.077 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-07-08T12:39:12.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:12.077 DEBUG:teuthology.orchestra.run.smithi012:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-07-08T12:39:12.122 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2023-07-08T12:39:12.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:12.123 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-08T12:39:12.141 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:39:12.141 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:39:12.163 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:39:12.163 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:39:12.222 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-07-08T12:39:12.223 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:39:12.282 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:39:12.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.405+0000 7f42bd675700 1 Processor -- start 2023-07-08T12:39:12.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.405+0000 7f42bd675700 1 -- start start 2023-07-08T12:39:12.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.406+0000 7f42bd675700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f42b80b4cc0 0x7f42b80cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:12.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.406+0000 7f42bd675700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b80cf3b0 con 0x7f42b80d1cd0 2023-07-08T12:39:12.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.406+0000 7f42bd675700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b80b86f0 con 0x7f42b80b8320 2023-07-08T12:39:12.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.406+0000 7f42bd675700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b80cf0d0 con 0x7f42b80b4cc0 2023-07-08T12:39:12.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.407+0000 7f42b77fe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f42b80d1cd0 0x7f42b80d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38490/0 (socket says 172.21.15.12:38490) 2023-07-08T12:39:12.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.407+0000 7f42b77fe700 1 -- 172.21.15.12:0/2838736146 learned_addr learned my addr 172.21.15.12:0/2838736146 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:12.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.407+0000 7f42b67fc700 1 --2- 172.21.15.12:0/2838736146 >> v2:172.21.15.160:3300/0 conn(0x7f42b80b4cc0 0x7f42b80cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:12.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.407+0000 7f42b5ffb700 1 -- 172.21.15.12:0/2838736146 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277691851 0 0) 0x7f42b80cf3b0 con 0x7f42b80d1cd0 2023-07-08T12:39:12.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.407+0000 7f42b5ffb700 1 -- 172.21.15.12:0/2838736146 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42a00035b0 con 0x7f42b80d1cd0 2023-07-08T12:39:12.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.407+0000 7f42b67fc700 1 -- 172.21.15.12:0/2838736146 >> v1:172.21.15.160:6789/0 conn(0x7f42b80b8320 legacy=0x7f42b80b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.407+0000 7f42b67fc700 1 -- 172.21.15.12:0/2838736146 >> v1:172.21.15.12:6789/0 conn(0x7f42b80d1cd0 legacy=0x7f42b80d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.407+0000 7f42b5ffb700 1 -- 172.21.15.12:0/2838736146 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1106608750 0 0) 0x7f42a00035b0 con 0x7f42b80d1cd0 2023-07-08T12:39:12.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.407+0000 7f42b67fc700 1 -- 172.21.15.12:0/2838736146 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b80d22c0 con 0x7f42b80b4cc0 2023-07-08T12:39:12.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.407+0000 7f42b67fc700 1 --2- 172.21.15.12:0/2838736146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42b80b4cc0 0x7f42b80cd4f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009840 tx=0x7f42ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af6f8e79cc2a5e69 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:12.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.407+0000 7f42b5ffb700 1 -- 172.21.15.12:0/2838736146 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b00019f0 con 0x7f42b80b4cc0 2023-07-08T12:39:12.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.407+0000 7f42b5ffb700 1 -- 172.21.15.12:0/2838736146 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac004030 con 0x7f42b80b4cc0 2023-07-08T12:39:12.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.407+0000 7f42b5ffb700 1 -- 172.21.15.12:0/2838736146 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0043a0 con 0x7f42b80b4cc0 2023-07-08T12:39:12.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.408+0000 7f42bd675700 1 -- 172.21.15.12:0/2838736146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42b80b4cc0 msgr2=0x7f42b80cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.408+0000 7f42bd675700 1 --2- 172.21.15.12:0/2838736146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42b80b4cc0 0x7f42b80cd4f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009840 tx=0x7f42ac009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:12.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.408+0000 7f42bd675700 1 -- 172.21.15.12:0/2838736146 shutdown_connections 2023-07-08T12:39:12.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.408+0000 7f42bd675700 1 --2- 172.21.15.12:0/2838736146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42b80b4cc0 0x7f42b80cd4f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.408+0000 7f42bd675700 1 -- 172.21.15.12:0/2838736146 >> 172.21.15.12:0/2838736146 conn(0x7f42b80722b0 msgr2=0x7f42b80cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:12.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.408+0000 7f42bd675700 1 -- 172.21.15.12:0/2838736146 shutdown_connections 2023-07-08T12:39:12.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.408+0000 7f42bd675700 1 -- 172.21.15.12:0/2838736146 wait complete. 2023-07-08T12:39:12.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.409+0000 7f42bd675700 1 Processor -- start 2023-07-08T12:39:12.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.409+0000 7f42bd675700 1 -- start start 2023-07-08T12:39:12.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.410+0000 7f42bd675700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42b80b8320 0x7f42b81f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:12.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.410+0000 7f42bd675700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42b80d1cd0 0x7f42b80a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:12.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.410+0000 7f42bd675700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42b80aa130 0x7f42b815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:12.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.410+0000 7f42bd675700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b815d610 con 0x7f42b80b8320 2023-07-08T12:39:12.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.410+0000 7f42bd675700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80aa5b0 con 0x7f42b80d1cd0 2023-07-08T12:39:12.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.410+0000 7f42bd675700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b815e430 con 0x7f42b80aa130 2023-07-08T12:39:12.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.410+0000 7f42b6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42b80b8320 0x7f42b81f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:12.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.410+0000 7f42b6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42b80b8320 0x7f42b81f1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51096/0 (socket says 172.21.15.12:51096) 2023-07-08T12:39:12.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.410+0000 7f42b6ffd700 1 -- 172.21.15.12:0/1388439179 learned_addr learned my addr 172.21.15.12:0/1388439179 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:12.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.410+0000 7f42b67fc700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42b80d1cd0 0x7f42b80a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.410+0000 7f42b77fe700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42b80aa130 0x7f42b815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.411+0000 7f42b6ffd700 1 -- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42b80aa130 msgr2=0x7f42b815e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.411+0000 7f42b6ffd700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42b80aa130 0x7f42b815e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.411+0000 7f42b6ffd700 1 -- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42b80d1cd0 msgr2=0x7f42b80a9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.411+0000 7f42b6ffd700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42b80d1cd0 0x7f42b80a9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.411+0000 7f42b6ffd700 1 -- 172.21.15.12:0/1388439179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac0094a0 con 0x7f42b80b8320 2023-07-08T12:39:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.411+0000 7f42b77fe700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42b80aa130 0x7f42b815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.411+0000 7f42b6ffd700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42b80b8320 0x7f42b81f1e60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f42a800c4e0 tx=0x7f42a800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.411+0000 7f429f7fe700 1 -- 172.21.15.12:0/1388439179 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a80052d0 con 0x7f42b80b8320 2023-07-08T12:39:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.411+0000 7f429f7fe700 1 -- 172.21.15.12:0/1388439179 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a8005430 con 0x7f42b80b8320 2023-07-08T12:39:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.411+0000 7f42bd675700 1 -- 172.21.15.12:0/1388439179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b815d820 con 0x7f42b80b8320 2023-07-08T12:39:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.411+0000 7f429f7fe700 1 -- 172.21.15.12:0/1388439179 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a80038c0 con 0x7f42b80b8320 2023-07-08T12:39:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.411+0000 7f42bd675700 1 -- 172.21.15.12:0/1388439179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b815dde0 con 0x7f42b80b8320 2023-07-08T12:39:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.412+0000 7f42bd675700 1 -- 172.21.15.12:0/1388439179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b804ea60 con 0x7f42b80b8320 2023-07-08T12:39:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.412+0000 7f429f7fe700 1 -- 172.21.15.12:0/1388439179 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f42a8003a20 con 0x7f42b80b8320 2023-07-08T12:39:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.413+0000 7f429f7fe700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f42a406ace0 0x7f42a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.413+0000 7f429f7fe700 1 -- 172.21.15.12:0/1388439179 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f42a8088c50 con 0x7f42b80b8320 2023-07-08T12:39:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.413+0000 7f42b67fc700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f42a406ace0 0x7f42a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.414+0000 7f42b67fc700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f42a406ace0 0x7f42a406d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009840 tx=0x7f42ac008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.418+0000 7f429f7fe700 1 -- 172.21.15.12:0/1388439179 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f42a808e020 con 0x7f42b80b8320 2023-07-08T12:39:12.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.623+0000 7f42bd675700 1 -- 172.21.15.12:0/1388439179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42b815b8c0 con 0x7f42b80b8320 2023-07-08T12:39:12.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.624+0000 7f429f7fe700 1 -- 172.21.15.12:0/1388439179 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20203 (secure 0 0 0) 0x7f42a80539d0 con 0x7f42b80b8320 2023-07-08T12:39:12.626 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:12.627 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:49.386462+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:39:12.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.626+0000 7f42bd675700 1 -- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f42a406ace0 msgr2=0x7f42a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.626+0000 7f42bd675700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f42a406ace0 0x7f42a406d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009840 tx=0x7f42ac008030 comp rx=0 tx=0).stop 2023-07-08T12:39:12.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.626+0000 7f42bd675700 1 -- 172.21.15.12:0/1388439179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42b80b8320 msgr2=0x7f42b81f1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.626+0000 7f42bd675700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42b80b8320 0x7f42b81f1e60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f42a800c4e0 tx=0x7f42a800c7f0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.626+0000 7f42bd675700 1 -- 172.21.15.12:0/1388439179 shutdown_connections 2023-07-08T12:39:12.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.626+0000 7f42bd675700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f42a406ace0 0x7f42a406d190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.626+0000 7f42bd675700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42b80aa130 0x7f42b815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.626+0000 7f42bd675700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42b80b8320 0x7f42b81f1e60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.626+0000 7f42bd675700 1 --2- 172.21.15.12:0/1388439179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42b80d1cd0 0x7f42b80a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.626+0000 7f42bd675700 1 -- 172.21.15.12:0/1388439179 >> 172.21.15.12:0/1388439179 conn(0x7f42b80b4cc0 msgr2=0x7f42b80b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:12.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.626+0000 7f42bd675700 1 -- 172.21.15.12:0/1388439179 shutdown_connections 2023-07-08T12:39:12.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.626+0000 7f42bd675700 1 -- 172.21.15.12:0/1388439179 wait complete. 2023-07-08T12:39:12.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-08T12:38:05.660194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-08T12:38:06.920177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-08T12:38:10.592749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-08T12:39:12.642 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:39:12.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.767+0000 7fbe0df25700 1 Processor -- start 2023-07-08T12:39:12.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.767+0000 7fbe0df25700 1 -- start start 2023-07-08T12:39:12.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.768+0000 7fbe0df25700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe0806e0c0 0x7fbe081608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:12.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.768+0000 7fbe0df25700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbe080cc240 0x7fbe080cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:12.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.768+0000 7fbe0df25700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe08160410 con 0x7fbe08156a10 2023-07-08T12:39:12.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.768+0000 7fbe0df25700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe080ccc00 con 0x7fbe080cc240 2023-07-08T12:39:12.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.768+0000 7fbe0df25700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe08156de0 con 0x7fbe0806e0c0 2023-07-08T12:39:12.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.768+0000 7fbe06ffd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe0806e0c0 0x7fbe081608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:12.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.768+0000 7fbe07fff700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbe080cc240 0x7fbe080cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:12.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.768+0000 7fbe06ffd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe0806e0c0 0x7fbe081608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:60724/0 (socket says 172.21.15.12:60724) 2023-07-08T12:39:12.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.768+0000 7fbe06ffd700 1 -- 172.21.15.12:0/4130898882 learned_addr learned my addr 172.21.15.12:0/4130898882 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:12.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.769+0000 7fbe06ffd700 1 -- 172.21.15.12:0/4130898882 >> v2:172.21.15.160:3300/0 conn(0x7fbe080cc240 msgr2=0x7fbe080cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:12.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.769+0000 7fbe06ffd700 1 --2- 172.21.15.12:0/4130898882 >> v2:172.21.15.160:3300/0 conn(0x7fbe080cc240 0x7fbe080cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.769+0000 7fbe06ffd700 1 -- 172.21.15.12:0/4130898882 >> v1:172.21.15.160:6789/0 conn(0x7fbe08156a10 legacy=0x7fbe0815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.769+0000 7fbe06ffd700 1 -- 172.21.15.12:0/4130898882 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0806ec70 con 0x7fbe0806e0c0 2023-07-08T12:39:12.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.769+0000 7fbe06ffd700 1 --2- 172.21.15.12:0/4130898882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe0806e0c0 0x7fbe081608a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbdf00098f0 tx=0x7fbdf0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e525323945776ca8 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:12.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.769+0000 7fbe067fc700 1 -- 172.21.15.12:0/4130898882 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0011830 con 0x7fbe0806e0c0 2023-07-08T12:39:12.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.769+0000 7fbe067fc700 1 -- 172.21.15.12:0/4130898882 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf0011990 con 0x7fbe0806e0c0 2023-07-08T12:39:12.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.769+0000 7fbe067fc700 1 -- 172.21.15.12:0/4130898882 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0011d20 con 0x7fbe0806e0c0 2023-07-08T12:39:12.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.770+0000 7fbe0df25700 1 -- 172.21.15.12:0/4130898882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe0806e0c0 msgr2=0x7fbe081608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.770+0000 7fbe0df25700 1 --2- 172.21.15.12:0/4130898882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe0806e0c0 0x7fbe081608a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbdf00098f0 tx=0x7fbdf0009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:12.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.770+0000 7fbe0df25700 1 -- 172.21.15.12:0/4130898882 shutdown_connections 2023-07-08T12:39:12.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.770+0000 7fbe0df25700 1 --2- 172.21.15.12:0/4130898882 >> v2:172.21.15.160:3300/0 conn(0x7fbe080cc240 0x7fbe080cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.770+0000 7fbe0df25700 1 --2- 172.21.15.12:0/4130898882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe0806e0c0 0x7fbe081608a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.770+0000 7fbe0df25700 1 -- 172.21.15.12:0/4130898882 >> 172.21.15.12:0/4130898882 conn(0x7fbe08098b50 msgr2=0x7fbe0806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:12.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.770+0000 7fbe0df25700 1 -- 172.21.15.12:0/4130898882 shutdown_connections 2023-07-08T12:39:12.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.770+0000 7fbe0df25700 1 -- 172.21.15.12:0/4130898882 wait complete. 2023-07-08T12:39:12.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.771+0000 7fbe0df25700 1 Processor -- start 2023-07-08T12:39:12.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.771+0000 7fbe0df25700 1 -- start start 2023-07-08T12:39:12.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe0df25700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe0806e0c0 0x7fbe08155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:12.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe0df25700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe080cc240 0x7fbe080b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:12.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe0df25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe08156a10 0x7fbe08157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:12.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe0df25700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080b4810 con 0x7fbe08156a10 2023-07-08T12:39:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe0df25700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe08155eb0 con 0x7fbe080cc240 2023-07-08T12:39:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe0df25700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080b3c00 con 0x7fbe0806e0c0 2023-07-08T12:39:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe08156a10 0x7fbe08157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe08156a10 0x7fbe08157450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51114/0 (socket says 172.21.15.12:51114) 2023-07-08T12:39:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe07fff700 1 -- 172.21.15.12:0/2909051602 learned_addr learned my addr 172.21.15.12:0/2909051602 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe06ffd700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe080cc240 0x7fbe080b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe077fe700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe0806e0c0 0x7fbe08155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe07fff700 1 -- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe0806e0c0 msgr2=0x7fbe08155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe07fff700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe0806e0c0 0x7fbe08155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe07fff700 1 -- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe080cc240 msgr2=0x7fbe080b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe07fff700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe080cc240 0x7fbe080b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.772+0000 7fbe07fff700 1 -- 172.21.15.12:0/2909051602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf0009550 con 0x7fbe08156a10 2023-07-08T12:39:12.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.773+0000 7fbe07fff700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe08156a10 0x7fbe08157450 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000efc0 tx=0x7fbe0000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:12.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.773+0000 7fbdfffff700 1 -- 172.21.15.12:0/2909051602 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00009ea0 con 0x7fbe08156a10 2023-07-08T12:39:12.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.773+0000 7fbdfffff700 1 -- 172.21.15.12:0/2909051602 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0000f040 con 0x7fbe08156a10 2023-07-08T12:39:12.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.773+0000 7fbe0df25700 1 -- 172.21.15.12:0/2909051602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe080b3ee0 con 0x7fbe08156a10 2023-07-08T12:39:12.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.773+0000 7fbdfffff700 1 -- 172.21.15.12:0/2909051602 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000ced0 con 0x7fbe08156a10 2023-07-08T12:39:12.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.773+0000 7fbe0df25700 1 -- 172.21.15.12:0/2909051602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe080b44a0 con 0x7fbe08156a10 2023-07-08T12:39:12.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.774+0000 7fbe0df25700 1 -- 172.21.15.12:0/2909051602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe080b22c0 con 0x7fbe08156a10 2023-07-08T12:39:12.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.774+0000 7fbdfffff700 1 -- 172.21.15.12:0/2909051602 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbe00014480 con 0x7fbe08156a10 2023-07-08T12:39:12.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.774+0000 7fbdfffff700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbde806ace0 0x7fbde806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:12.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.775+0000 7fbdfffff700 1 -- 172.21.15.12:0/2909051602 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbe0008a0f0 con 0x7fbe08156a10 2023-07-08T12:39:12.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.775+0000 7fbe077fe700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbde806ace0 0x7fbde806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:12.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.775+0000 7fbe077fe700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbde806ace0 0x7fbde806d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbe080aebd0 tx=0x7fbdf8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:12.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.779+0000 7fbdfffff700 1 -- 172.21.15.12:0/2909051602 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbe00054df0 con 0x7fbe08156a10 2023-07-08T12:39:12.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.975+0000 7fbe0df25700 1 -- 172.21.15.12:0/2909051602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe0804ea60 con 0x7fbe08156a10 2023-07-08T12:39:12.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.975+0000 7fbdfffff700 1 -- 172.21.15.12:0/2909051602 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe00058410 con 0x7fbe08156a10 2023-07-08T12:39:12.978 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:39:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.977+0000 7fbe0df25700 1 -- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbde806ace0 msgr2=0x7fbde806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.977+0000 7fbe0df25700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbde806ace0 0x7fbde806d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbe080aebd0 tx=0x7fbdf8009040 comp rx=0 tx=0).stop 2023-07-08T12:39:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.977+0000 7fbe0df25700 1 -- 172.21.15.12:0/2909051602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe08156a10 msgr2=0x7fbe08157450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.977+0000 7fbe0df25700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe08156a10 0x7fbe08157450 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000efc0 tx=0x7fbe0000c3a0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.977+0000 7fbe0df25700 1 -- 172.21.15.12:0/2909051602 shutdown_connections 2023-07-08T12:39:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.977+0000 7fbe0df25700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fbde806ace0 0x7fbde806d190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.977+0000 7fbe0df25700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe08156a10 0x7fbe08157450 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.977+0000 7fbe0df25700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe0806e0c0 0x7fbe08155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.977+0000 7fbe0df25700 1 --2- 172.21.15.12:0/2909051602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe080cc240 0x7fbe080b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.977+0000 7fbe0df25700 1 -- 172.21.15.12:0/2909051602 >> 172.21.15.12:0/2909051602 conn(0x7fbe08098b50 msgr2=0x7fbe0806cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.977+0000 7fbe0df25700 1 -- 172.21.15.12:0/2909051602 shutdown_connections 2023-07-08T12:39:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:12.977+0000 7fbe0df25700 1 -- 172.21.15.12:0/2909051602 wait complete. 2023-07-08T12:39:12.991 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:39:13.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.118+0000 7fe3a950f700 1 Processor -- start 2023-07-08T12:39:13.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.118+0000 7fe3a950f700 1 -- start start 2023-07-08T12:39:13.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.119+0000 7fe3a950f700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe3a40d3730 0x7fe3a40cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.119+0000 7fe3a950f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3a40cfa90 0x7fe3a40d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.119+0000 7fe3a950f700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a40d0b20 con 0x7fe3a40b8430 2023-07-08T12:39:13.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.119+0000 7fe3a950f700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a40cda30 con 0x7fe3a40d3730 2023-07-08T12:39:13.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.119+0000 7fe3a950f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40b8800 con 0x7fe3a40cfa90 2023-07-08T12:39:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.119+0000 7fe3a27fc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe3a40d3730 0x7fe3a40cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.119+0000 7fe3a27fc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe3a40d3730 0x7fe3a40cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51120/0 (socket says 172.21.15.12:51120) 2023-07-08T12:39:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.119+0000 7fe3a27fc700 1 -- 172.21.15.12:0/4240060693 learned_addr learned my addr 172.21.15.12:0/4240060693 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.119+0000 7fe3a37fe700 1 --2- 172.21.15.12:0/4240060693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3a40cfa90 0x7fe3a40d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.119+0000 7fe3a27fc700 1 -- 172.21.15.12:0/4240060693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3a40cfa90 msgr2=0x7fe3a40d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.119+0000 7fe3a27fc700 1 --2- 172.21.15.12:0/4240060693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3a40cfa90 0x7fe3a40d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.119+0000 7fe3a27fc700 1 -- 172.21.15.12:0/4240060693 >> v1:172.21.15.160:6789/0 conn(0x7fe3a40b8430 legacy=0x7fe3a40cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.119+0000 7fe3a27fc700 1 -- 172.21.15.12:0/4240060693 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a40cfeb0 con 0x7fe3a40d3730 2023-07-08T12:39:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.120+0000 7fe3a27fc700 1 --2- 172.21.15.12:0/4240060693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a40d3730 0x7fe3a40cf550 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe3940098f0 tx=0x7fe394009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8fb91fa147a6b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.120+0000 7fe3a1ffb700 1 -- 172.21.15.12:0/4240060693 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39400b8a0 con 0x7fe3a40d3730 2023-07-08T12:39:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.120+0000 7fe3a1ffb700 1 -- 172.21.15.12:0/4240060693 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39400ba00 con 0x7fe3a40d3730 2023-07-08T12:39:13.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.120+0000 7fe3a1ffb700 1 -- 172.21.15.12:0/4240060693 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39400bd90 con 0x7fe3a40d3730 2023-07-08T12:39:13.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.120+0000 7fe3a950f700 1 -- 172.21.15.12:0/4240060693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a40d3730 msgr2=0x7fe3a40cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.120+0000 7fe3a950f700 1 --2- 172.21.15.12:0/4240060693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a40d3730 0x7fe3a40cf550 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe3940098f0 tx=0x7fe394009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:13.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.120+0000 7fe3a950f700 1 -- 172.21.15.12:0/4240060693 shutdown_connections 2023-07-08T12:39:13.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.120+0000 7fe3a950f700 1 --2- 172.21.15.12:0/4240060693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3a40cfa90 0x7fe3a40d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.120+0000 7fe3a950f700 1 --2- 172.21.15.12:0/4240060693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a40d3730 0x7fe3a40cf550 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.120+0000 7fe3a950f700 1 -- 172.21.15.12:0/4240060693 >> 172.21.15.12:0/4240060693 conn(0x7fe3a406b3f0 msgr2=0x7fe3a406b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:13.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.120+0000 7fe3a950f700 1 -- 172.21.15.12:0/4240060693 shutdown_connections 2023-07-08T12:39:13.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.120+0000 7fe3a950f700 1 -- 172.21.15.12:0/4240060693 wait complete. 2023-07-08T12:39:13.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.122+0000 7fe3a950f700 1 Processor -- start 2023-07-08T12:39:13.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.122+0000 7fe3a950f700 1 -- start start 2023-07-08T12:39:13.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.122+0000 7fe3a950f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a40b8430 0x7fe3a41f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a950f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3a40cfa90 0x7fe3a40b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a950f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3a40d3730 0x7fe3a40cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a950f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41f2c10 con 0x7fe3a40b8430 2023-07-08T12:39:13.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a950f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41f2090 con 0x7fe3a40d3730 2023-07-08T12:39:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a950f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41f2370 con 0x7fe3a40cfa90 2023-07-08T12:39:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a40b8430 0x7fe3a41f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a40b8430 0x7fe3a41f2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51124/0 (socket says 172.21.15.12:51124) 2023-07-08T12:39:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a2ffd700 1 -- 172.21.15.12:0/3708019299 learned_addr learned my addr 172.21.15.12:0/3708019299 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a27fc700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3a40cfa90 0x7fe3a40b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a37fe700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3a40d3730 0x7fe3a40cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a2ffd700 1 -- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3a40cfa90 msgr2=0x7fe3a40b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a2ffd700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3a40cfa90 0x7fe3a40b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a2ffd700 1 -- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3a40d3730 msgr2=0x7fe3a40cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a2ffd700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3a40d3730 0x7fe3a40cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a2ffd700 1 -- 172.21.15.12:0/3708019299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe394009550 con 0x7fe3a40b8430 2023-07-08T12:39:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a27fc700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3a40cfa90 0x7fe3a40b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a2ffd700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a40b8430 0x7fe3a41f2d70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe39800c4e0 tx=0x7fe39800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe38b7fe700 1 -- 172.21.15.12:0/3708019299 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980052d0 con 0x7fe3a40b8430 2023-07-08T12:39:13.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe38b7fe700 1 -- 172.21.15.12:0/3708019299 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe398005430 con 0x7fe3a40b8430 2023-07-08T12:39:13.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a950f700 1 -- 172.21.15.12:0/3708019299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a41f2650 con 0x7fe3a40b8430 2023-07-08T12:39:13.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe38b7fe700 1 -- 172.21.15.12:0/3708019299 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980038c0 con 0x7fe3a40b8430 2023-07-08T12:39:13.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.123+0000 7fe3a950f700 1 -- 172.21.15.12:0/3708019299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a41f32b0 con 0x7fe3a40b8430 2023-07-08T12:39:13.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.124+0000 7fe3a950f700 1 -- 172.21.15.12:0/3708019299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a404ea60 con 0x7fe3a40b8430 2023-07-08T12:39:13.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.125+0000 7fe38b7fe700 1 -- 172.21.15.12:0/3708019299 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe398003a20 con 0x7fe3a40b8430 2023-07-08T12:39:13.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.125+0000 7fe38b7fe700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe39006ace0 0x7fe39006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.125+0000 7fe38b7fe700 1 -- 172.21.15.12:0/3708019299 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe398088cd0 con 0x7fe3a40b8430 2023-07-08T12:39:13.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.125+0000 7fe3a27fc700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe39006ace0 0x7fe39006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.126+0000 7fe3a27fc700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe39006ace0 0x7fe39006d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe394014040 tx=0x7fe394008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:13.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.130+0000 7fe38b7fe700 1 -- 172.21.15.12:0/3708019299 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe398053a50 con 0x7fe3a40b8430 2023-07-08T12:39:13.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.327+0000 7fe3a950f700 1 -- 172.21.15.12:0/3708019299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe3a40d1530 con 0x7fe3a40b8430 2023-07-08T12:39:13.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.327+0000 7fe38b7fe700 1 -- 172.21.15.12:0/3708019299 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe398017020 con 0x7fe3a40b8430 2023-07-08T12:39:13.330 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:39:13.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.329+0000 7fe3a950f700 1 -- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe39006ace0 msgr2=0x7fe39006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.329+0000 7fe3a950f700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe39006ace0 0x7fe39006d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe394014040 tx=0x7fe394008030 comp rx=0 tx=0).stop 2023-07-08T12:39:13.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.329+0000 7fe3a950f700 1 -- 172.21.15.12:0/3708019299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a40b8430 msgr2=0x7fe3a41f2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.329+0000 7fe3a950f700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a40b8430 0x7fe3a41f2d70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe39800c4e0 tx=0x7fe39800c8a0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.329+0000 7fe3a950f700 1 -- 172.21.15.12:0/3708019299 shutdown_connections 2023-07-08T12:39:13.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.329+0000 7fe3a950f700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe39006ace0 0x7fe39006d190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.329+0000 7fe3a950f700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3a40d3730 0x7fe3a40cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.329+0000 7fe3a950f700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a40b8430 0x7fe3a41f2d70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.329+0000 7fe3a950f700 1 --2- 172.21.15.12:0/3708019299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3a40cfa90 0x7fe3a40b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.329+0000 7fe3a950f700 1 -- 172.21.15.12:0/3708019299 >> 172.21.15.12:0/3708019299 conn(0x7fe3a406b3f0 msgr2=0x7fe3a40d1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:13.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.329+0000 7fe3a950f700 1 -- 172.21.15.12:0/3708019299 shutdown_connections 2023-07-08T12:39:13.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.329+0000 7fe3a950f700 1 -- 172.21.15.12:0/3708019299 wait complete. 2023-07-08T12:39:13.343 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-08T12:39:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.470+0000 7f7eb5103700 1 Processor -- start 2023-07-08T12:39:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.470+0000 7f7eb5103700 1 -- start start 2023-07-08T12:39:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eb5103700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7eb00cdb80 0x7f7eb00cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eb5103700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb015d770 con 0x7f7eb00aeec0 2023-07-08T12:39:13.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eb5103700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb015cb60 con 0x7f7eb006ebe0 2023-07-08T12:39:13.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eb5103700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb006d700 con 0x7f7eb00cdb80 2023-07-08T12:39:13.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eaf59e700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7eb00cdb80 0x7f7eb00cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eaf59e700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7eb00cdb80 0x7f7eb00cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51136/0 (socket says 172.21.15.12:51136) 2023-07-08T12:39:13.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eaf59e700 1 -- 172.21.15.12:0/602109983 learned_addr learned my addr 172.21.15.12:0/602109983 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:13.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eaed9d700 1 --1- 172.21.15.12:0/602109983 >> v1:172.21.15.12:6789/0 conn(0x7f7eb00aeec0 0x7f7eb006cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38528/0 (socket says 172.21.15.12:38528) 2023-07-08T12:39:13.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eaf59e700 1 -- 172.21.15.12:0/602109983 >> v1:172.21.15.160:6789/0 conn(0x7f7eb006ebe0 legacy=0x7f7eb00cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:13.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eaf59e700 1 -- 172.21.15.12:0/602109983 >> v1:172.21.15.12:6789/0 conn(0x7f7eb00aeec0 legacy=0x7f7eb006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eaf59e700 1 -- 172.21.15.12:0/602109983 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb015ce40 con 0x7f7eb00cdb80 2023-07-08T12:39:13.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eaf59e700 1 --2- 172.21.15.12:0/602109983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb00cdb80 0x7f7eb00cbe10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800d150 tx=0x7f7ea800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d499ca562701e96c server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:13.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eadd9b700 1 -- 172.21.15.12:0/602109983 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8010070 con 0x7f7eb00cdb80 2023-07-08T12:39:13.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eadd9b700 1 -- 172.21.15.12:0/602109983 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea800dd80 con 0x7f7eb00cdb80 2023-07-08T12:39:13.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.471+0000 7f7eadd9b700 1 -- 172.21.15.12:0/602109983 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800b310 con 0x7f7eb00cdb80 2023-07-08T12:39:13.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.472+0000 7f7eb5103700 1 -- 172.21.15.12:0/602109983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb00cdb80 msgr2=0x7f7eb00cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.472+0000 7f7eb5103700 1 --2- 172.21.15.12:0/602109983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb00cdb80 0x7f7eb00cbe10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800d150 tx=0x7f7ea800d460 comp rx=0 tx=0).stop 2023-07-08T12:39:13.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.472+0000 7f7eb5103700 1 -- 172.21.15.12:0/602109983 shutdown_connections 2023-07-08T12:39:13.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.472+0000 7f7eb5103700 1 --2- 172.21.15.12:0/602109983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb00cdb80 0x7f7eb00cbe10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.472+0000 7f7eb5103700 1 -- 172.21.15.12:0/602109983 >> 172.21.15.12:0/602109983 conn(0x7f7eb0098b50 msgr2=0x7f7eb006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:13.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.472+0000 7f7eb5103700 1 -- 172.21.15.12:0/602109983 shutdown_connections 2023-07-08T12:39:13.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.472+0000 7f7eb5103700 1 -- 172.21.15.12:0/602109983 wait complete. 2023-07-08T12:39:13.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.473+0000 7f7eb5103700 1 Processor -- start 2023-07-08T12:39:13.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.473+0000 7f7eb5103700 1 -- start start 2023-07-08T12:39:13.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.474+0000 7f7eb5103700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7eb006ebe0 0x7f7eb01583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.474+0000 7f7eb5103700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb00aeec0 0x7f7eb00c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.474+0000 7f7eb5103700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eb00cdb80 0x7f7eb00cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.474+0000 7f7eb5103700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb01588e0 con 0x7f7eb00aeec0 2023-07-08T12:39:13.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.474+0000 7f7eb5103700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb00b15e0 con 0x7f7eb00cdb80 2023-07-08T12:39:13.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.474+0000 7f7eb5103700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb00b18c0 con 0x7f7eb006ebe0 2023-07-08T12:39:13.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.474+0000 7f7eae59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb00aeec0 0x7f7eb00c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.474+0000 7f7eae59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb00aeec0 0x7f7eb00c5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51140/0 (socket says 172.21.15.12:51140) 2023-07-08T12:39:13.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.474+0000 7f7eae59c700 1 -- 172.21.15.12:0/360624890 learned_addr learned my addr 172.21.15.12:0/360624890 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:13.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.474+0000 7f7eaed9d700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7eb006ebe0 0x7f7eb01583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.474+0000 7f7eaf59e700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eb00cdb80 0x7f7eb00cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.475+0000 7f7eae59c700 1 -- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7eb006ebe0 msgr2=0x7f7eb01583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.475+0000 7f7eae59c700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7eb006ebe0 0x7f7eb01583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.475+0000 7f7eae59c700 1 -- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eb00cdb80 msgr2=0x7f7eb00cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.475+0000 7f7eae59c700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eb00cdb80 0x7f7eb00cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.475+0000 7f7eae59c700 1 -- 172.21.15.12:0/360624890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea800bd40 con 0x7f7eb00aeec0 2023-07-08T12:39:13.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.475+0000 7f7eaed9d700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7eb006ebe0 0x7f7eb01583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:13.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.475+0000 7f7eaf59e700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eb00cdb80 0x7f7eb00cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:13.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.475+0000 7f7eae59c700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb00aeec0 0x7f7eb00c5ab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000c4d0 tx=0x7f7ea000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:13.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.475+0000 7f7e977fe700 1 -- 172.21.15.12:0/360624890 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0011790 con 0x7f7eb00aeec0 2023-07-08T12:39:13.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.475+0000 7f7e977fe700 1 -- 172.21.15.12:0/360624890 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea00118f0 con 0x7f7eb00aeec0 2023-07-08T12:39:13.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.475+0000 7f7eb5103700 1 -- 172.21.15.12:0/360624890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb00b1ba0 con 0x7f7eb00aeec0 2023-07-08T12:39:13.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.475+0000 7f7e977fe700 1 -- 172.21.15.12:0/360624890 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0011ca0 con 0x7f7eb00aeec0 2023-07-08T12:39:13.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.475+0000 7f7eb5103700 1 -- 172.21.15.12:0/360624890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb00c5ff0 con 0x7f7eb00aeec0 2023-07-08T12:39:13.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.476+0000 7f7eb5103700 1 -- 172.21.15.12:0/360624890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb004ea60 con 0x7f7eb00aeec0 2023-07-08T12:39:13.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.477+0000 7f7e977fe700 1 -- 172.21.15.12:0/360624890 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7ea001c460 con 0x7f7eb00aeec0 2023-07-08T12:39:13.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.477+0000 7f7e977fe700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7e9c06ad10 0x7f7e9c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.477+0000 7f7e977fe700 1 -- 172.21.15.12:0/360624890 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7ea00895f0 con 0x7f7eb00aeec0 2023-07-08T12:39:13.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.477+0000 7f7eaed9d700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7e9c06ad10 0x7f7e9c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.478+0000 7f7eaed9d700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7e9c06ad10 0x7f7e9c06d1c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4001020 tx=0x7f7ea4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:13.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.481+0000 7f7e977fe700 1 -- 172.21.15.12:0/360624890 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7ea0054370 con 0x7f7eb00aeec0 2023-07-08T12:39:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.683+0000 7f7eb5103700 1 -- 172.21.15.12:0/360624890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7eb00ccde0 con 0x7f7eb00aeec0 2023-07-08T12:39:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.684+0000 7f7e977fe700 1 -- 172.21.15.12:0/360624890 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7ea0057990 con 0x7f7eb00aeec0 2023-07-08T12:39:13.686 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:13.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.686+0000 7f7eb5103700 1 -- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7e9c06ad10 msgr2=0x7f7e9c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.686+0000 7f7eb5103700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7e9c06ad10 0x7f7e9c06d1c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4001020 tx=0x7f7ea4009040 comp rx=0 tx=0).stop 2023-07-08T12:39:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.686+0000 7f7eb5103700 1 -- 172.21.15.12:0/360624890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb00aeec0 msgr2=0x7f7eb00c5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.686+0000 7f7eb5103700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb00aeec0 0x7f7eb00c5ab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000c4d0 tx=0x7f7ea000c890 comp rx=0 tx=0).stop 2023-07-08T12:39:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.686+0000 7f7eb5103700 1 -- 172.21.15.12:0/360624890 shutdown_connections 2023-07-08T12:39:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.686+0000 7f7eb5103700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7e9c06ad10 0x7f7e9c06d1c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.686+0000 7f7eb5103700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eb00cdb80 0x7f7eb00cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.686+0000 7f7eb5103700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7eb006ebe0 0x7f7eb01583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.686+0000 7f7eb5103700 1 --2- 172.21.15.12:0/360624890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb00aeec0 0x7f7eb00c5ab0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.686+0000 7f7eb5103700 1 -- 172.21.15.12:0/360624890 >> 172.21.15.12:0/360624890 conn(0x7f7eb0098b50 msgr2=0x7f7eb0159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:13.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.686+0000 7f7eb5103700 1 -- 172.21.15.12:0/360624890 shutdown_connections 2023-07-08T12:39:13.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.686+0000 7f7eb5103700 1 -- 172.21.15.12:0/360624890 wait complete. 2023-07-08T12:39:13.700 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-08T12:39:13.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.826+0000 7f4a6a987700 1 Processor -- start 2023-07-08T12:39:13.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.826+0000 7f4a6a987700 1 -- start start 2023-07-08T12:39:13.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a6a987700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a6415a6d0 0x7f4a6415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a6a987700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4a64161ce0 0x7f4a641640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a6a987700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a64158130 con 0x7f4a640ce560 2023-07-08T12:39:13.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a6a987700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6415b730 con 0x7f4a6415a6d0 2023-07-08T12:39:13.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a6a987700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6415b450 con 0x7f4a64161ce0 2023-07-08T12:39:13.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a637fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a6415a6d0 0x7f4a6415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a63fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4a640ce560 0x7f4a640ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38542/0 (socket says 172.21.15.12:38542) 2023-07-08T12:39:13.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a637fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a6415a6d0 0x7f4a6415ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51148/0 (socket says 172.21.15.12:51148) 2023-07-08T12:39:13.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a63fff700 1 -- 172.21.15.12:0/4262453652 learned_addr learned my addr 172.21.15.12:0/4262453652 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:13.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a637fe700 1 -- 172.21.15.12:0/4262453652 >> v2:172.21.15.160:3300/0 conn(0x7f4a64161ce0 msgr2=0x7f4a641640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:13.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a68f24700 1 --2- 172.21.15.12:0/4262453652 >> v2:172.21.15.160:3300/0 conn(0x7f4a64161ce0 0x7f4a641640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a637fe700 1 --2- 172.21.15.12:0/4262453652 >> v2:172.21.15.160:3300/0 conn(0x7f4a64161ce0 0x7f4a641640c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a62ffd700 1 -- 172.21.15.12:0/4262453652 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333963846 0 0) 0x7f4a64158130 con 0x7f4a640ce560 2023-07-08T12:39:13.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a637fe700 1 -- 172.21.15.12:0/4262453652 >> v1:172.21.15.12:6789/0 conn(0x7f4a640ce560 legacy=0x7f4a640ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.827+0000 7f4a637fe700 1 -- 172.21.15.12:0/4262453652 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6406e850 con 0x7f4a6415a6d0 2023-07-08T12:39:13.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.828+0000 7f4a637fe700 1 --2- 172.21.15.12:0/4262453652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a6415a6d0 0x7f4a6415ab20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4a640cf120 tx=0x7f4a54009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d66729fac6a705 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:13.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.828+0000 7f4a62ffd700 1 -- 172.21.15.12:0/4262453652 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54011830 con 0x7f4a6415a6d0 2023-07-08T12:39:13.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.828+0000 7f4a62ffd700 1 -- 172.21.15.12:0/4262453652 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a54011990 con 0x7f4a6415a6d0 2023-07-08T12:39:13.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.828+0000 7f4a62ffd700 1 -- 172.21.15.12:0/4262453652 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54011d20 con 0x7f4a6415a6d0 2023-07-08T12:39:13.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.828+0000 7f4a6a987700 1 -- 172.21.15.12:0/4262453652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a6415a6d0 msgr2=0x7f4a6415ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.828+0000 7f4a6a987700 1 --2- 172.21.15.12:0/4262453652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a6415a6d0 0x7f4a6415ab20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4a640cf120 tx=0x7f4a54009b20 comp rx=0 tx=0).stop 2023-07-08T12:39:13.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.828+0000 7f4a6a987700 1 -- 172.21.15.12:0/4262453652 shutdown_connections 2023-07-08T12:39:13.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.828+0000 7f4a6a987700 1 --2- 172.21.15.12:0/4262453652 >> v2:172.21.15.160:3300/0 conn(0x7f4a64161ce0 0x7f4a641640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.828+0000 7f4a6a987700 1 --2- 172.21.15.12:0/4262453652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a6415a6d0 0x7f4a6415ab20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.828+0000 7f4a6a987700 1 -- 172.21.15.12:0/4262453652 >> 172.21.15.12:0/4262453652 conn(0x7f4a64099c60 msgr2=0x7f4a64089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:13.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.828+0000 7f4a6a987700 1 -- 172.21.15.12:0/4262453652 shutdown_connections 2023-07-08T12:39:13.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.828+0000 7f4a6a987700 1 -- 172.21.15.12:0/4262453652 wait complete. 2023-07-08T12:39:13.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.830+0000 7f4a6a987700 1 Processor -- start 2023-07-08T12:39:13.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.830+0000 7f4a6a987700 1 -- start start 2023-07-08T12:39:13.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.831+0000 7f4a6a987700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a640ce560 0x7f4a640af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.831+0000 7f4a6a987700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a6415a6d0 0x7f4a64156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.831+0000 7f4a6a987700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a64161ce0 0x7f4a6414acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.831+0000 7f4a6a987700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a640ad550 con 0x7f4a640ce560 2023-07-08T12:39:13.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.831+0000 7f4a6a987700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a640ad830 con 0x7f4a64161ce0 2023-07-08T12:39:13.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.831+0000 7f4a6a987700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a640adb10 con 0x7f4a6415a6d0 2023-07-08T12:39:13.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.831+0000 7f4a63fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a640ce560 0x7f4a640af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.831+0000 7f4a63fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a640ce560 0x7f4a640af260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51152/0 (socket says 172.21.15.12:51152) 2023-07-08T12:39:13.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.831+0000 7f4a63fff700 1 -- 172.21.15.12:0/3756373015 learned_addr learned my addr 172.21.15.12:0/3756373015 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:13.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.831+0000 7f4a68f24700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a64161ce0 0x7f4a6414acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.831+0000 7f4a637fe700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a6415a6d0 0x7f4a64156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.831+0000 7f4a63fff700 1 -- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a6415a6d0 msgr2=0x7f4a64156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.832+0000 7f4a63fff700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a6415a6d0 0x7f4a64156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.832+0000 7f4a63fff700 1 -- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a64161ce0 msgr2=0x7f4a6414acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:13.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.832+0000 7f4a63fff700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a64161ce0 0x7f4a6414acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:13.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.832+0000 7f4a63fff700 1 -- 172.21.15.12:0/3756373015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a540094a0 con 0x7f4a640ce560 2023-07-08T12:39:13.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.832+0000 7f4a68f24700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a64161ce0 0x7f4a6414acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:13.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.832+0000 7f4a637fe700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a6415a6d0 0x7f4a64156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:13.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.832+0000 7f4a63fff700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a640ce560 0x7f4a640af260 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c550 tx=0x7f4a5800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:13.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.832+0000 7f4a60ff9700 1 -- 172.21.15.12:0/3756373015 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58004c80 con 0x7f4a640ce560 2023-07-08T12:39:13.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.832+0000 7f4a60ff9700 1 -- 172.21.15.12:0/3756373015 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a58004e00 con 0x7f4a640ce560 2023-07-08T12:39:13.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.832+0000 7f4a6a987700 1 -- 172.21.15.12:0/3756373015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a640addf0 con 0x7f4a640ce560 2023-07-08T12:39:13.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.832+0000 7f4a60ff9700 1 -- 172.21.15.12:0/3756373015 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58007310 con 0x7f4a640ce560 2023-07-08T12:39:13.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.832+0000 7f4a6a987700 1 -- 172.21.15.12:0/3756373015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a640ae5d0 con 0x7f4a640ce560 2023-07-08T12:39:13.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.833+0000 7f4a6a987700 1 -- 172.21.15.12:0/3756373015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6404ea60 con 0x7f4a640ce560 2023-07-08T12:39:13.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.833+0000 7f4a60ff9700 1 -- 172.21.15.12:0/3756373015 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a58007470 con 0x7f4a640ce560 2023-07-08T12:39:13.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.834+0000 7f4a60ff9700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a5006ace0 0x7f4a5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:13.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.834+0000 7f4a60ff9700 1 -- 172.21.15.12:0/3756373015 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4a580894c0 con 0x7f4a640ce560 2023-07-08T12:39:13.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.834+0000 7f4a637fe700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a5006ace0 0x7f4a5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:13.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.835+0000 7f4a637fe700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a5006ace0 0x7f4a5006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4a6414c490 tx=0x7f4a54008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:13.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:13.838+0000 7f4a60ff9700 1 -- 172.21.15.12:0/3756373015 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4a58054240 con 0x7f4a640ce560 2023-07-08T12:39:14.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.036+0000 7f4a6a987700 1 -- 172.21.15.12:0/3756373015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a640ae440 con 0x7f4a640ce560 2023-07-08T12:39:14.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.036+0000 7f4a60ff9700 1 -- 172.21.15.12:0/3756373015 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a58057860 con 0x7f4a640ce560 2023-07-08T12:39:14.039 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-07-08T12:39:14.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.038+0000 7f4a6a987700 1 -- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a5006ace0 msgr2=0x7f4a5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.038+0000 7f4a6a987700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a5006ace0 0x7f4a5006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4a6414c490 tx=0x7f4a54008030 comp rx=0 tx=0).stop 2023-07-08T12:39:14.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.038+0000 7f4a6a987700 1 -- 172.21.15.12:0/3756373015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a640ce560 msgr2=0x7f4a640af260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.038+0000 7f4a6a987700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a640ce560 0x7f4a640af260 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c550 tx=0x7f4a5800c910 comp rx=0 tx=0).stop 2023-07-08T12:39:14.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.038+0000 7f4a6a987700 1 -- 172.21.15.12:0/3756373015 shutdown_connections 2023-07-08T12:39:14.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.038+0000 7f4a6a987700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f4a5006ace0 0x7f4a5006d190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.038+0000 7f4a6a987700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a64161ce0 0x7f4a6414acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.038+0000 7f4a6a987700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a640ce560 0x7f4a640af260 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.038+0000 7f4a6a987700 1 --2- 172.21.15.12:0/3756373015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a6415a6d0 0x7f4a64156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.038+0000 7f4a6a987700 1 -- 172.21.15.12:0/3756373015 >> 172.21.15.12:0/3756373015 conn(0x7f4a64099c60 msgr2=0x7f4a6406f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.038+0000 7f4a6a987700 1 -- 172.21.15.12:0/3756373015 shutdown_connections 2023-07-08T12:39:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.038+0000 7f4a6a987700 1 -- 172.21.15.12:0/3756373015 wait complete. 2023-07-08T12:39:14.053 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-08T12:39:14.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.178+0000 7f90d67b3700 1 Processor -- start 2023-07-08T12:39:14.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.178+0000 7f90d67b3700 1 -- start start 2023-07-08T12:39:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90d67b3700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90d00aeec0 0x7f90d006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90d67b3700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f90d00cdb80 0x7f90d00cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90d67b3700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d015d770 con 0x7f90d006ebe0 2023-07-08T12:39:14.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90d67b3700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d015a600 con 0x7f90d00cdb80 2023-07-08T12:39:14.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90d67b3700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d015cb60 con 0x7f90d00aeec0 2023-07-08T12:39:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90cffff700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90d00aeec0 0x7f90d006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90cffff700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90d00aeec0 0x7f90d006cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:60776/0 (socket says 172.21.15.12:60776) 2023-07-08T12:39:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90d4d50700 1 --2- 172.21.15.12:0/2959633276 >> v2:172.21.15.160:3300/0 conn(0x7f90d00cdb80 0x7f90d00cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90cffff700 1 -- 172.21.15.12:0/2959633276 learned_addr learned my addr 172.21.15.12:0/2959633276 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:14.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90cffff700 1 -- 172.21.15.12:0/2959633276 >> v2:172.21.15.160:3300/0 conn(0x7f90d00cdb80 msgr2=0x7f90d00cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:14.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90cffff700 1 --2- 172.21.15.12:0/2959633276 >> v2:172.21.15.160:3300/0 conn(0x7f90d00cdb80 0x7f90d00cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90cffff700 1 -- 172.21.15.12:0/2959633276 >> v1:172.21.15.160:6789/0 conn(0x7f90d006ebe0 legacy=0x7f90d00cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90cffff700 1 -- 172.21.15.12:0/2959633276 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d015ce40 con 0x7f90d00aeec0 2023-07-08T12:39:14.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.179+0000 7f90d4d50700 1 --2- 172.21.15.12:0/2959633276 >> v2:172.21.15.160:3300/0 conn(0x7f90d00cdb80 0x7f90d00cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:39:14.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.180+0000 7f90cffff700 1 --2- 172.21.15.12:0/2959633276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90d00aeec0 0x7f90d006cff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f90c0006fd0 tx=0x7f90c0009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f918d79d3787bdad server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:14.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.180+0000 7f90ceffd700 1 -- 172.21.15.12:0/2959633276 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c0005e30 con 0x7f90d00aeec0 2023-07-08T12:39:14.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.180+0000 7f90ceffd700 1 -- 172.21.15.12:0/2959633276 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c0004030 con 0x7f90d00aeec0 2023-07-08T12:39:14.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.180+0000 7f90ceffd700 1 -- 172.21.15.12:0/2959633276 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c00043e0 con 0x7f90d00aeec0 2023-07-08T12:39:14.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.180+0000 7f90d67b3700 1 -- 172.21.15.12:0/2959633276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90d00aeec0 msgr2=0x7f90d006cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.180+0000 7f90d67b3700 1 --2- 172.21.15.12:0/2959633276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90d00aeec0 0x7f90d006cff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f90c0006fd0 tx=0x7f90c0009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.180+0000 7f90d67b3700 1 -- 172.21.15.12:0/2959633276 shutdown_connections 2023-07-08T12:39:14.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.180+0000 7f90d67b3700 1 --2- 172.21.15.12:0/2959633276 >> v2:172.21.15.160:3300/0 conn(0x7f90d00cdb80 0x7f90d00cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.180+0000 7f90d67b3700 1 --2- 172.21.15.12:0/2959633276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90d00aeec0 0x7f90d006cff0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.180+0000 7f90d67b3700 1 -- 172.21.15.12:0/2959633276 >> 172.21.15.12:0/2959633276 conn(0x7f90d0098b50 msgr2=0x7f90d006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:14.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.180+0000 7f90d67b3700 1 -- 172.21.15.12:0/2959633276 shutdown_connections 2023-07-08T12:39:14.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.180+0000 7f90d67b3700 1 -- 172.21.15.12:0/2959633276 wait complete. 2023-07-08T12:39:14.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.182+0000 7f90d67b3700 1 Processor -- start 2023-07-08T12:39:14.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.182+0000 7f90d67b3700 1 -- start start 2023-07-08T12:39:14.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.182+0000 7f90d67b3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d006ebe0 0x7f90d00b5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.182+0000 7f90d67b3700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90d00aeec0 0x7f90d00c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.182+0000 7f90d67b3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90d00cdb80 0x7f90d0154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90d67b3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d00c4dc0 con 0x7f90d006ebe0 2023-07-08T12:39:14.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90d67b3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d00c6090 con 0x7f90d00cdb80 2023-07-08T12:39:14.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90d67b3700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d00c4ff0 con 0x7f90d00aeec0 2023-07-08T12:39:14.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90cffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d006ebe0 0x7f90d00b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90cffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d006ebe0 0x7f90d00b5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51164/0 (socket says 172.21.15.12:51164) 2023-07-08T12:39:14.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90cffff700 1 -- 172.21.15.12:0/1673884388 learned_addr learned my addr 172.21.15.12:0/1673884388 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:14.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90cf7fe700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90d00aeec0 0x7f90d00c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90d4d50700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90d00cdb80 0x7f90d0154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90cffff700 1 -- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90d00aeec0 msgr2=0x7f90d00c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90cffff700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90d00aeec0 0x7f90d00c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90cffff700 1 -- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90d00cdb80 msgr2=0x7f90d0154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90cffff700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90d00cdb80 0x7f90d0154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90cffff700 1 -- 172.21.15.12:0/1673884388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c8009a70 con 0x7f90d006ebe0 2023-07-08T12:39:14.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90cffff700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d006ebe0 0x7f90d00b5230 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f90c0000c00 tx=0x7f90c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:14.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90ccff9700 1 -- 172.21.15.12:0/1673884388 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c0003dd0 con 0x7f90d006ebe0 2023-07-08T12:39:14.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90ccff9700 1 -- 172.21.15.12:0/1673884388 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c001c420 con 0x7f90d006ebe0 2023-07-08T12:39:14.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90d67b3700 1 -- 172.21.15.12:0/1673884388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c0009500 con 0x7f90d006ebe0 2023-07-08T12:39:14.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90ccff9700 1 -- 172.21.15.12:0/1673884388 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c001c7b0 con 0x7f90d006ebe0 2023-07-08T12:39:14.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.183+0000 7f90d67b3700 1 -- 172.21.15.12:0/1673884388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d00c55b0 con 0x7f90d006ebe0 2023-07-08T12:39:14.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.184+0000 7f90d67b3700 1 -- 172.21.15.12:0/1673884388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d004ea60 con 0x7f90d006ebe0 2023-07-08T12:39:14.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.185+0000 7f90ccff9700 1 -- 172.21.15.12:0/1673884388 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f90c001c910 con 0x7f90d006ebe0 2023-07-08T12:39:14.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.185+0000 7f90ccff9700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f90bc06ace0 0x7f90bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.185+0000 7f90ccff9700 1 -- 172.21.15.12:0/1673884388 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f90c008a860 con 0x7f90d006ebe0 2023-07-08T12:39:14.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.186+0000 7f90cf7fe700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f90bc06ace0 0x7f90bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.186+0000 7f90cf7fe700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f90bc06ace0 0x7f90bc06d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f90d006e130 tx=0x7f90c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:14.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.190+0000 7f90ccff9700 1 -- 172.21.15.12:0/1673884388 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f90c0055670 con 0x7f90d006ebe0 2023-07-08T12:39:14.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.389+0000 7f90d67b3700 1 -- 172.21.15.12:0/1673884388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f90d0066af0 con 0x7f90d006ebe0 2023-07-08T12:39:14.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.390+0000 7f90ccff9700 1 -- 172.21.15.12:0/1673884388 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f90c0058c90 con 0x7f90d006ebe0 2023-07-08T12:39:14.392 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:14.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.392+0000 7f90d67b3700 1 -- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f90bc06ace0 msgr2=0x7f90bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.392+0000 7f90d67b3700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f90bc06ace0 0x7f90bc06d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f90d006e130 tx=0x7f90c4009040 comp rx=0 tx=0).stop 2023-07-08T12:39:14.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.392+0000 7f90d67b3700 1 -- 172.21.15.12:0/1673884388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d006ebe0 msgr2=0x7f90d00b5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.392+0000 7f90d67b3700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d006ebe0 0x7f90d00b5230 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f90c0000c00 tx=0x7f90c0004750 comp rx=0 tx=0).stop 2023-07-08T12:39:14.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.392+0000 7f90d67b3700 1 -- 172.21.15.12:0/1673884388 shutdown_connections 2023-07-08T12:39:14.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.392+0000 7f90d67b3700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f90bc06ace0 0x7f90bc06d190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.392+0000 7f90d67b3700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90d00cdb80 0x7f90d0154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.392+0000 7f90d67b3700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d006ebe0 0x7f90d00b5230 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.392+0000 7f90d67b3700 1 --2- 172.21.15.12:0/1673884388 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90d00aeec0 0x7f90d00c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.392+0000 7f90d67b3700 1 -- 172.21.15.12:0/1673884388 >> 172.21.15.12:0/1673884388 conn(0x7f90d0098b50 msgr2=0x7f90d00ade90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:14.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.392+0000 7f90d67b3700 1 -- 172.21.15.12:0/1673884388 shutdown_connections 2023-07-08T12:39:14.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.392+0000 7f90d67b3700 1 -- 172.21.15.12:0/1673884388 wait complete. 2023-07-08T12:39:14.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-08T12:39:14.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.531+0000 7f579166d700 1 Processor -- start 2023-07-08T12:39:14.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.531+0000 7f579166d700 1 -- start start 2023-07-08T12:39:14.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f579166d700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f578c15a190 0x7f578c15e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f579166d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f578c0cd940 0x7f578c0cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f579166d700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f578c0ce4b0 con 0x7f578c0cb130 2023-07-08T12:39:14.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f579166d700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f578c15ec40 con 0x7f578c15a190 2023-07-08T12:39:14.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f579166d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578c0ccc70 con 0x7f578c0cd940 2023-07-08T12:39:14.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f578affd700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f578c15a190 0x7f578c15e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f578b7fe700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f578c0cd940 0x7f578c0cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f578a7fc700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f578c0cb130 0x7f578c0cd230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:47792/0 (socket says 172.21.15.12:47792) 2023-07-08T12:39:14.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f578a7fc700 1 -- 172.21.15.12:0/3666708456 learned_addr learned my addr 172.21.15.12:0/3666708456 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:14.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f578b7fe700 1 -- 172.21.15.12:0/3666708456 >> v2:172.21.15.160:3300/0 conn(0x7f578c15a190 msgr2=0x7f578c15e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:14.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f578b7fe700 1 --2- 172.21.15.12:0/3666708456 >> v2:172.21.15.160:3300/0 conn(0x7f578c15a190 0x7f578c15e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f578b7fe700 1 -- 172.21.15.12:0/3666708456 >> v1:172.21.15.160:6789/0 conn(0x7f578c0cb130 legacy=0x7f578c0cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f578b7fe700 1 -- 172.21.15.12:0/3666708456 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c0ccf50 con 0x7f578c0cd940 2023-07-08T12:39:14.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f5789ffb700 1 -- 172.21.15.12:0/3666708456 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799449398 0 0) 0x7f578c0ce4b0 con 0x7f578c0cb130 2023-07-08T12:39:14.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.532+0000 7f578affd700 1 --2- 172.21.15.12:0/3666708456 >> v2:172.21.15.160:3300/0 conn(0x7f578c15a190 0x7f578c15e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:39:14.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.533+0000 7f578b7fe700 1 --2- 172.21.15.12:0/3666708456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f578c0cd940 0x7f578c0cef00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5784009d90 tx=0x7f578400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b1eddf962839690 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:14.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.533+0000 7f5789ffb700 1 -- 172.21.15.12:0/3666708456 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578400e070 con 0x7f578c0cd940 2023-07-08T12:39:14.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.533+0000 7f5789ffb700 1 -- 172.21.15.12:0/3666708456 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578400bb20 con 0x7f578c0cd940 2023-07-08T12:39:14.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.533+0000 7f5789ffb700 1 -- 172.21.15.12:0/3666708456 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784012430 con 0x7f578c0cd940 2023-07-08T12:39:14.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.533+0000 7f579166d700 1 -- 172.21.15.12:0/3666708456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f578c0cd940 msgr2=0x7f578c0cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.533+0000 7f579166d700 1 --2- 172.21.15.12:0/3666708456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f578c0cd940 0x7f578c0cef00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5784009d90 tx=0x7f578400b330 comp rx=0 tx=0).stop 2023-07-08T12:39:14.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.533+0000 7f579166d700 1 -- 172.21.15.12:0/3666708456 shutdown_connections 2023-07-08T12:39:14.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.533+0000 7f579166d700 1 --2- 172.21.15.12:0/3666708456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f578c0cd940 0x7f578c0cef00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.533+0000 7f579166d700 1 --2- 172.21.15.12:0/3666708456 >> v2:172.21.15.160:3300/0 conn(0x7f578c15a190 0x7f578c15e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.533+0000 7f579166d700 1 -- 172.21.15.12:0/3666708456 >> 172.21.15.12:0/3666708456 conn(0x7f578c06e550 msgr2=0x7f578c06e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:14.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.533+0000 7f579166d700 1 -- 172.21.15.12:0/3666708456 shutdown_connections 2023-07-08T12:39:14.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.533+0000 7f579166d700 1 -- 172.21.15.12:0/3666708456 wait complete. 2023-07-08T12:39:14.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.535+0000 7f579166d700 1 Processor -- start 2023-07-08T12:39:14.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.535+0000 7f579166d700 1 -- start start 2023-07-08T12:39:14.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f579166d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c0cb130 0x7f578c15b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f579166d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578c0cd940 0x7f578c0c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f579166d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f578c15a190 0x7f578c0b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f579166d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c15cb30 con 0x7f578c0cd940 2023-07-08T12:39:14.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f579166d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c15bf20 con 0x7f578c0cb130 2023-07-08T12:39:14.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f579166d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578c15c200 con 0x7f578c15a190 2023-07-08T12:39:14.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f578a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578c0cd940 0x7f578c0c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f578a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578c0cd940 0x7f578c0c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51180/0 (socket says 172.21.15.12:51180) 2023-07-08T12:39:14.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f578a7fc700 1 -- 172.21.15.12:0/2055255639 learned_addr learned my addr 172.21.15.12:0/2055255639 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:14.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f578b7fe700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f578c15a190 0x7f578c0b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f578affd700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c0cb130 0x7f578c15b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f578a7fc700 1 -- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f578c15a190 msgr2=0x7f578c0b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f578a7fc700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f578c15a190 0x7f578c0b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f578a7fc700 1 -- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c0cb130 msgr2=0x7f578c15b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f578a7fc700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c0cb130 0x7f578c15b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f578a7fc700 1 -- 172.21.15.12:0/2055255639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0094a0 con 0x7f578c0cd940 2023-07-08T12:39:14.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f578a7fc700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578c0cd940 0x7f578c0c6e30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f577400c550 tx=0x7f577400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:14.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f57837fe700 1 -- 172.21.15.12:0/2055255639 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774004c80 con 0x7f578c0cd940 2023-07-08T12:39:14.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f57837fe700 1 -- 172.21.15.12:0/2055255639 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5774004e00 con 0x7f578c0cd940 2023-07-08T12:39:14.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f579166d700 1 -- 172.21.15.12:0/2055255639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5784009a70 con 0x7f578c0cd940 2023-07-08T12:39:14.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f57837fe700 1 -- 172.21.15.12:0/2055255639 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774007310 con 0x7f578c0cd940 2023-07-08T12:39:14.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.536+0000 7f579166d700 1 -- 172.21.15.12:0/2055255639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578c15c7c0 con 0x7f578c0cd940 2023-07-08T12:39:14.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.537+0000 7f579166d700 1 -- 172.21.15.12:0/2055255639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578c15cca0 con 0x7f578c0cd940 2023-07-08T12:39:14.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.538+0000 7f57837fe700 1 -- 172.21.15.12:0/2055255639 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5774007470 con 0x7f578c0cd940 2023-07-08T12:39:14.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.538+0000 7f57837fe700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f576c06ace0 0x7f576c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.538+0000 7f57837fe700 1 -- 172.21.15.12:0/2055255639 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f57740895b0 con 0x7f578c0cd940 2023-07-08T12:39:14.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.539+0000 7f578affd700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f576c06ace0 0x7f576c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.539+0000 7f578affd700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f576c06ace0 0x7f576c06d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f577c009d00 tx=0x7f577c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:14.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.543+0000 7f57837fe700 1 -- 172.21.15.12:0/2055255639 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5774054290 con 0x7f578c0cd940 2023-07-08T12:39:14.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.757+0000 7f579166d700 1 -- 172.21.15.12:0/2055255639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f578c04ea60 con 0x7f578c0cd940 2023-07-08T12:39:14.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.757+0000 7f57837fe700 1 -- 172.21.15.12:0/2055255639 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f57740578b0 con 0x7f578c0cd940 2023-07-08T12:39:14.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.759+0000 7f579166d700 1 -- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f576c06ace0 msgr2=0x7f576c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.759+0000 7f579166d700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f576c06ace0 0x7f576c06d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f577c009d00 tx=0x7f577c008040 comp rx=0 tx=0).stop 2023-07-08T12:39:14.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.759+0000 7f579166d700 1 -- 172.21.15.12:0/2055255639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578c0cd940 msgr2=0x7f578c0c6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.759+0000 7f579166d700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578c0cd940 0x7f578c0c6e30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f577400c550 tx=0x7f577400c910 comp rx=0 tx=0).stop 2023-07-08T12:39:14.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.759+0000 7f579166d700 1 -- 172.21.15.12:0/2055255639 shutdown_connections 2023-07-08T12:39:14.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.759+0000 7f579166d700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f576c06ace0 0x7f576c06d190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.759+0000 7f579166d700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f578c15a190 0x7f578c0b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.759+0000 7f579166d700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f578c0cb130 0x7f578c15b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.759+0000 7f579166d700 1 --2- 172.21.15.12:0/2055255639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f578c0cd940 0x7f578c0c6e30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.759+0000 7f579166d700 1 -- 172.21.15.12:0/2055255639 >> 172.21.15.12:0/2055255639 conn(0x7f578c06e550 msgr2=0x7f578c0cfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:14.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.759+0000 7f579166d700 1 -- 172.21.15.12:0/2055255639 shutdown_connections 2023-07-08T12:39:14.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.759+0000 7f579166d700 1 -- 172.21.15.12:0/2055255639 wait complete. 2023-07-08T12:39:14.763 INFO:teuthology.orchestra.run.smithi012.stderr:listed 0 entries 2023-07-08T12:39:14.773 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-07-08T12:39:14.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:14.774 DEBUG:teuthology.orchestra.run.smithi160:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-07-08T12:39:14.819 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi160.front.sepia.ceph.com 2023-07-08T12:39:14.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:14.820 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-08T12:39:14.839 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-08T12:39:14.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-08T12:39:14.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.963+0000 7fd5de6b7700 1 Processor -- start 2023-07-08T12:39:14.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.963+0000 7fd5de6b7700 1 -- start start 2023-07-08T12:39:14.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.964+0000 7fd5de6b7700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd5d80b1c70 0x7fd5d80ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.964+0000 7fd5de6b7700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d815d230 con 0x7fd5d80ce1b0 2023-07-08T12:39:14.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.964+0000 7fd5de6b7700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d80ceac0 con 0x7fd5d815d6c0 2023-07-08T12:39:14.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.964+0000 7fd5de6b7700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d80c9e60 con 0x7fd5d80b1c70 2023-07-08T12:39:14.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.964+0000 7fd5d7fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd5d80ce1b0 0x7fd5d80cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38582/0 (socket says 172.21.15.12:38582) 2023-07-08T12:39:14.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.964+0000 7fd5d7fff700 1 -- 172.21.15.12:0/3327542790 learned_addr learned my addr 172.21.15.12:0/3327542790 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:14.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d77fe700 1 --2- 172.21.15.12:0/3327542790 >> v2:172.21.15.160:3300/0 conn(0x7fd5d80b1c70 0x7fd5d80ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445617398 0 0) 0x7fd5d815d230 con 0x7fd5d80ce1b0 2023-07-08T12:39:14.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5c00035b0 con 0x7fd5d80ce1b0 2023-07-08T12:39:14.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1274076018 0 0) 0x7fd5c00035b0 con 0x7fd5d80ce1b0 2023-07-08T12:39:14.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 >> v2:172.21.15.160:3300/0 conn(0x7fd5d80b1c70 msgr2=0x7fd5d80ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:14.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 --2- 172.21.15.12:0/3327542790 >> v2:172.21.15.160:3300/0 conn(0x7fd5d80b1c70 0x7fd5d80ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 >> v1:172.21.15.160:6789/0 conn(0x7fd5d815d6c0 legacy=0x7fd5d8071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d80cdbc0 con 0x7fd5d80ce1b0 2023-07-08T12:39:14.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7fd5c8001670 con 0x7fd5d80ce1b0 2023-07-08T12:39:14.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 >> v1:172.21.15.12:6789/0 conn(0x7fd5d80ce1b0 legacy=0x7fd5d80cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 --2- 172.21.15.12:0/3327542790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5c0003a00 0x7fd5c0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 --2- 172.21.15.12:0/3327542790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5c0004850 0x7fd5c0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 --2- 172.21.15.12:0/3327542790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5c0005200 0x7fd5c000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c000c0e0 con 0x7fd5c0004850 2023-07-08T12:39:14.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c000c250 con 0x7fd5c0005200 2023-07-08T12:39:14.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c000c410 con 0x7fd5c0003a00 2023-07-08T12:39:14.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d77fe700 1 --2- 172.21.15.12:0/3327542790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5c0004850 0x7fd5c0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5dcc54700 1 --2- 172.21.15.12:0/3327542790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5c0005200 0x7fd5c000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d7fff700 1 --2- 172.21.15.12:0/3327542790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5c0003a00 0x7fd5c0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d77fe700 1 -- 172.21.15.12:0/3327542790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5c0003a00 msgr2=0x7fd5c0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:14.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d77fe700 1 --2- 172.21.15.12:0/3327542790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5c0003a00 0x7fd5c0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d77fe700 1 -- 172.21.15.12:0/3327542790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5c0005200 msgr2=0x7fd5c000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:14.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d77fe700 1 --2- 172.21.15.12:0/3327542790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5c0005200 0x7fd5c000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5d77fe700 1 -- 172.21.15.12:0/3327542790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c00035b0 con 0x7fd5c0004850 2023-07-08T12:39:14.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5dcc54700 1 --2- 172.21.15.12:0/3327542790 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5c0005200 0x7fd5c000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:14.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.965+0000 7fd5dcc54700 1 -- 172.21.15.12:0/3327542790 reap_dead start 2023-07-08T12:39:14.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.966+0000 7fd5d77fe700 1 --2- 172.21.15.12:0/3327542790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5c0004850 0x7fd5c0004cc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009b60 tx=0x7fd5c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aea037fc7714953 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:14.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.966+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc00e6c0 con 0x7fd5c0004850 2023-07-08T12:39:14.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.966+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5cc00e820 con 0x7fd5c0004850 2023-07-08T12:39:14.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.966+0000 7fd5d6ffd700 1 -- 172.21.15.12:0/3327542790 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc00eb90 con 0x7fd5c0004850 2023-07-08T12:39:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.966+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3327542790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5c0004850 msgr2=0x7fd5c0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.966+0000 7fd5de6b7700 1 --2- 172.21.15.12:0/3327542790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5c0004850 0x7fd5c0004cc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009b60 tx=0x7fd5c0005880 comp rx=0 tx=0).stop 2023-07-08T12:39:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.966+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3327542790 shutdown_connections 2023-07-08T12:39:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.966+0000 7fd5de6b7700 1 --2- 172.21.15.12:0/3327542790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5c0004850 0x7fd5c0004cc0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.966+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3327542790 >> 172.21.15.12:0/3327542790 conn(0x7fd5d80cc290 msgr2=0x7fd5d80cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.966+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3327542790 shutdown_connections 2023-07-08T12:39:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.966+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3327542790 wait complete. 2023-07-08T12:39:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.968+0000 7fd5de6b7700 1 Processor -- start 2023-07-08T12:39:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.968+0000 7fd5de6b7700 1 -- start start 2023-07-08T12:39:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.968+0000 7fd5de6b7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5c0004850 0x7fd5d8158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.968+0000 7fd5de6b7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5d80c5d70 0x7fd5d80ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.968+0000 7fd5de6b7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5d81599b0 0x7fd5d8159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5de6b7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d815a360 con 0x7fd5c0004850 2023-07-08T12:39:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5de6b7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d80accc0 con 0x7fd5d80c5d70 2023-07-08T12:39:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5de6b7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d80ac140 con 0x7fd5d81599b0 2023-07-08T12:39:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5d7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5c0004850 0x7fd5d8158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5d7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5c0004850 0x7fd5d8158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51198/0 (socket says 172.21.15.12:51198) 2023-07-08T12:39:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5d7fff700 1 -- 172.21.15.12:0/3530573178 learned_addr learned my addr 172.21.15.12:0/3530573178 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5d7fff700 1 -- 172.21.15.12:0/3530573178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5d81599b0 msgr2=0x7fd5d8159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5d7fff700 1 --2- 172.21.15.12:0/3530573178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5d81599b0 0x7fd5d8159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5d7fff700 1 -- 172.21.15.12:0/3530573178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5d80c5d70 msgr2=0x7fd5d80ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5d7fff700 1 --2- 172.21.15.12:0/3530573178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5d80c5d70 0x7fd5d80ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5d7fff700 1 -- 172.21.15.12:0/3530573178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d80ac420 con 0x7fd5c0004850 2023-07-08T12:39:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5d7fff700 1 --2- 172.21.15.12:0/3530573178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5c0004850 0x7fd5d8158050 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e8f0 tx=0x7fd5c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5d4ff9700 1 -- 172.21.15.12:0/3530573178 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800c700 con 0x7fd5c0004850 2023-07-08T12:39:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5d4ff9700 1 -- 172.21.15.12:0/3530573178 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c800c860 con 0x7fd5c0004850 2023-07-08T12:39:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3530573178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d80ac700 con 0x7fd5c0004850 2023-07-08T12:39:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5d4ff9700 1 -- 172.21.15.12:0/3530573178 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800cbf0 con 0x7fd5c0004850 2023-07-08T12:39:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.969+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3530573178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d8158590 con 0x7fd5c0004850 2023-07-08T12:39:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.970+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3530573178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d80acf20 con 0x7fd5c0004850 2023-07-08T12:39:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.971+0000 7fd5d4ff9700 1 -- 172.21.15.12:0/3530573178 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd5c800cd50 con 0x7fd5c0004850 2023-07-08T12:39:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.971+0000 7fd5d4ff9700 1 --2- 172.21.15.12:0/3530573178 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd5c406ace0 0x7fd5c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.971+0000 7fd5d4ff9700 1 -- 172.21.15.12:0/3530573178 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd5c8089590 con 0x7fd5c0004850 2023-07-08T12:39:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.971+0000 7fd5d77fe700 1 --2- 172.21.15.12:0/3530573178 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd5c406ace0 0x7fd5c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.972+0000 7fd5d77fe700 1 --2- 172.21.15.12:0/3530573178 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd5c406ace0 0x7fd5c406d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80af4c0 tx=0x7fd5cc013000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:14.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:14.975+0000 7fd5d4ff9700 1 -- 172.21.15.12:0/3530573178 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd5d80acf20 con 0x7fd5c0004850 2023-07-08T12:39:15.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.220+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3530573178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5d80acf20 con 0x7fd5c0004850 2023-07-08T12:39:15.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.221+0000 7fd5d4ff9700 1 -- 172.21.15.12:0/3530573178 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fd5d80acf20 con 0x7fd5c0004850 2023-07-08T12:39:15.223 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:15.224 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3238006310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3238006310},{"type":"v1","addr":"172.21.15.160:6836","nonce":3238006310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3864661454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3864661454},{"type":"v1","addr":"172.21.15.160:6837","nonce":3864661454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6834/946118830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":946118830},{"type":"v1","addr":"172.21.15.12:6834","nonce":946118830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-08T12:38:07.943202+0000","modified":"2023-07-08T12:38:32.319568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6835/3301134992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3301134992},{"type":"v1","addr":"172.21.15.12:6835","nonce":3301134992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-08T12:39:15.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.223+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3530573178 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd5c406ace0 msgr2=0x7fd5c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.223+0000 7fd5de6b7700 1 --2- 172.21.15.12:0/3530573178 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd5c406ace0 0x7fd5c406d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80af4c0 tx=0x7fd5cc013000 comp rx=0 tx=0).stop 2023-07-08T12:39:15.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.223+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3530573178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5c0004850 msgr2=0x7fd5d8158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.223+0000 7fd5de6b7700 1 --2- 172.21.15.12:0/3530573178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5c0004850 0x7fd5d8158050 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e8f0 tx=0x7fd5c800ecb0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.223+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3530573178 shutdown_connections 2023-07-08T12:39:15.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.223+0000 7fd5de6b7700 1 --2- 172.21.15.12:0/3530573178 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd5c406ace0 0x7fd5c406d190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.223+0000 7fd5de6b7700 1 --2- 172.21.15.12:0/3530573178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5d81599b0 0x7fd5d8159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.223+0000 7fd5de6b7700 1 --2- 172.21.15.12:0/3530573178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5c0004850 0x7fd5d8158050 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.223+0000 7fd5de6b7700 1 --2- 172.21.15.12:0/3530573178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5d80c5d70 0x7fd5d80ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.223+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3530573178 >> 172.21.15.12:0/3530573178 conn(0x7fd5d80cc290 msgr2=0x7fd5d80af810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:15.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.223+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3530573178 shutdown_connections 2023-07-08T12:39:15.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.223+0000 7fd5de6b7700 1 -- 172.21.15.12:0/3530573178 wait complete. 2023-07-08T12:39:15.228 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2023-07-08T12:39:15.238 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:39:15.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.362+0000 7fa0a62be700 1 Processor -- start 2023-07-08T12:39:15.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.362+0000 7fa0a62be700 1 -- start start 2023-07-08T12:39:15.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.363+0000 7fa0a62be700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa0a006a640 0x7fa0a00cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:15.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.363+0000 7fa0a62be700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a015c890 con 0x7fa0a0159c30 2023-07-08T12:39:15.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.363+0000 7fa0a62be700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a006f620 con 0x7fa0a0160790 2023-07-08T12:39:15.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.363+0000 7fa0a62be700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a015c730 con 0x7fa0a006a640 2023-07-08T12:39:15.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.363+0000 7fa0a485b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa0a006a640 0x7fa0a00cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:15.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.363+0000 7fa09ffff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa0a0159c30 0x7fa0a006ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38602/0 (socket says 172.21.15.12:38602) 2023-07-08T12:39:15.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.363+0000 7fa09ffff700 1 -- 172.21.15.12:0/1382717653 learned_addr learned my addr 172.21.15.12:0/1382717653 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:15.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.364+0000 7fa0a485b700 1 -- 172.21.15.12:0/1382717653 >> v1:172.21.15.160:6789/0 conn(0x7fa0a0160790 legacy=0x7fa0a0069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.364+0000 7fa0a485b700 1 -- 172.21.15.12:0/1382717653 >> v1:172.21.15.12:6789/0 conn(0x7fa0a0159c30 legacy=0x7fa0a006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.364+0000 7fa09effd700 1 -- 172.21.15.12:0/1382717653 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334830265 0 0) 0x7fa0a015c890 con 0x7fa0a0159c30 2023-07-08T12:39:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.364+0000 7fa0a485b700 1 -- 172.21.15.12:0/1382717653 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a0160c90 con 0x7fa0a006a640 2023-07-08T12:39:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.364+0000 7fa0a485b700 1 --2- 172.21.15.12:0/1382717653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0a006a640 0x7fa0a00cf050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa098009d90 tx=0x7fa09800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8c8dc5392f8613 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.364+0000 7fa09effd700 1 -- 172.21.15.12:0/1382717653 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09800e070 con 0x7fa0a006a640 2023-07-08T12:39:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.364+0000 7fa09effd700 1 -- 172.21.15.12:0/1382717653 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09800bb90 con 0x7fa0a006a640 2023-07-08T12:39:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.364+0000 7fa09effd700 1 -- 172.21.15.12:0/1382717653 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098003a20 con 0x7fa0a006a640 2023-07-08T12:39:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.364+0000 7fa0a62be700 1 -- 172.21.15.12:0/1382717653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0a006a640 msgr2=0x7fa0a00cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.364+0000 7fa0a62be700 1 --2- 172.21.15.12:0/1382717653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0a006a640 0x7fa0a00cf050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa098009d90 tx=0x7fa09800b330 comp rx=0 tx=0).stop 2023-07-08T12:39:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.364+0000 7fa0a62be700 1 -- 172.21.15.12:0/1382717653 shutdown_connections 2023-07-08T12:39:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.364+0000 7fa0a62be700 1 --2- 172.21.15.12:0/1382717653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0a006a640 0x7fa0a00cf050 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.364+0000 7fa0a62be700 1 -- 172.21.15.12:0/1382717653 >> 172.21.15.12:0/1382717653 conn(0x7fa0a00cc290 msgr2=0x7fa0a00cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.365+0000 7fa0a62be700 1 -- 172.21.15.12:0/1382717653 shutdown_connections 2023-07-08T12:39:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.365+0000 7fa0a62be700 1 -- 172.21.15.12:0/1382717653 wait complete. 2023-07-08T12:39:15.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.366+0000 7fa0a62be700 1 Processor -- start 2023-07-08T12:39:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.366+0000 7fa0a62be700 1 -- start start 2023-07-08T12:39:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa0a62be700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0a006a640 0x7fa0a014d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa0a62be700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0a0159c30 0x7fa0a00ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa0a62be700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0a0160790 0x7fa0a014dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa0a62be700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00ad170 con 0x7fa0a0159c30 2023-07-08T12:39:15.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa0a62be700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00ac560 con 0x7fa0a006a640 2023-07-08T12:39:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa0a62be700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00ac840 con 0x7fa0a0160790 2023-07-08T12:39:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0a0159c30 0x7fa0a00ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0a0159c30 0x7fa0a00ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51214/0 (socket says 172.21.15.12:51214) 2023-07-08T12:39:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09f7fe700 1 -- 172.21.15.12:0/1961178700 learned_addr learned my addr 172.21.15.12:0/1961178700 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09ffff700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0a006a640 0x7fa0a014d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa0a485b700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0a0160790 0x7fa0a014dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09f7fe700 1 -- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0a0160790 msgr2=0x7fa0a014dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09f7fe700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0a0160790 0x7fa0a014dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09f7fe700 1 -- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0a006a640 msgr2=0x7fa0a014d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09f7fe700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0a006a640 0x7fa0a014d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09f7fe700 1 -- 172.21.15.12:0/1961178700 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa098009a70 con 0x7fa0a0159c30 2023-07-08T12:39:15.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09ffff700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0a006a640 0x7fa0a014d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:15.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09f7fe700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0a0159c30 0x7fa0a00ad600 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa08800c4a0 tx=0x7fa08800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:15.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09cff9700 1 -- 172.21.15.12:0/1961178700 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088011790 con 0x7fa0a0159c30 2023-07-08T12:39:15.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09cff9700 1 -- 172.21.15.12:0/1961178700 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0880118f0 con 0x7fa0a0159c30 2023-07-08T12:39:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa0a62be700 1 -- 172.21.15.12:0/1961178700 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a00acb20 con 0x7fa0a0159c30 2023-07-08T12:39:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.367+0000 7fa09cff9700 1 -- 172.21.15.12:0/1961178700 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088011ca0 con 0x7fa0a0159c30 2023-07-08T12:39:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.368+0000 7fa0a62be700 1 -- 172.21.15.12:0/1961178700 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a014d8a0 con 0x7fa0a0159c30 2023-07-08T12:39:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.368+0000 7fa0a62be700 1 -- 172.21.15.12:0/1961178700 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a004ea60 con 0x7fa0a0159c30 2023-07-08T12:39:15.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.369+0000 7fa09cff9700 1 -- 172.21.15.12:0/1961178700 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa08801c460 con 0x7fa0a0159c30 2023-07-08T12:39:15.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.369+0000 7fa09cff9700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa08006ace0 0x7fa08006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:15.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.369+0000 7fa09cff9700 1 -- 172.21.15.12:0/1961178700 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa088089700 con 0x7fa0a0159c30 2023-07-08T12:39:15.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.369+0000 7fa09ffff700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa08006ace0 0x7fa08006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:15.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.370+0000 7fa09ffff700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa08006ace0 0x7fa08006d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa090000f60 tx=0x7fa090007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:15.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.374+0000 7fa09cff9700 1 -- 172.21.15.12:0/1961178700 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa088054350 con 0x7fa0a0159c30 2023-07-08T12:39:15.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.574+0000 7fa0a62be700 1 -- 172.21.15.12:0/1961178700 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0a00b5790 con 0x7fa0a0159c30 2023-07-08T12:39:15.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.575+0000 7fa09cff9700 1 -- 172.21.15.12:0/1961178700 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20203 (secure 0 0 0) 0x7fa088057970 con 0x7fa0a0159c30 2023-07-08T12:39:15.578 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:15.579 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:49.386462+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:39:15.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.577+0000 7fa0a62be700 1 -- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa08006ace0 msgr2=0x7fa08006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.577+0000 7fa0a62be700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa08006ace0 0x7fa08006d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa090000f60 tx=0x7fa090007ca0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.577+0000 7fa0a62be700 1 -- 172.21.15.12:0/1961178700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0a0159c30 msgr2=0x7fa0a00ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.577+0000 7fa0a62be700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0a0159c30 0x7fa0a00ad600 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa08800c4a0 tx=0x7fa08800c860 comp rx=0 tx=0).stop 2023-07-08T12:39:15.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.577+0000 7fa0a62be700 1 -- 172.21.15.12:0/1961178700 shutdown_connections 2023-07-08T12:39:15.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.577+0000 7fa0a62be700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa08006ace0 0x7fa08006d190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.577+0000 7fa0a62be700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0a0160790 0x7fa0a014dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.577+0000 7fa0a62be700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0a006a640 0x7fa0a014d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.577+0000 7fa0a62be700 1 --2- 172.21.15.12:0/1961178700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0a0159c30 0x7fa0a00ad600 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.577+0000 7fa0a62be700 1 -- 172.21.15.12:0/1961178700 >> 172.21.15.12:0/1961178700 conn(0x7fa0a00cc290 msgr2=0x7fa0a0069f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:15.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.577+0000 7fa0a62be700 1 -- 172.21.15.12:0/1961178700 shutdown_connections 2023-07-08T12:39:15.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.578+0000 7fa0a62be700 1 -- 172.21.15.12:0/1961178700 wait complete. 2023-07-08T12:39:15.593 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-08T12:38:05.660194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-08T12:38:06.920177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-08T12:38:10.592749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-08T12:39:15.594 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:39:15.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.719+0000 7f7ec3984700 1 Processor -- start 2023-07-08T12:39:15.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.719+0000 7f7ec3984700 1 -- start start 2023-07-08T12:39:15.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.720+0000 7f7ec3984700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ebc156a10 0x7f7ebc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:15.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.720+0000 7f7ec3984700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7ebc15d390 0x7f7ebc1608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:15.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.720+0000 7f7ec3984700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ebc160410 con 0x7f7ebc0cc240 2023-07-08T12:39:15.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.720+0000 7f7ec3984700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ebc156de0 con 0x7f7ebc15d390 2023-07-08T12:39:15.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.720+0000 7f7ec3984700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc06ec70 con 0x7f7ebc156a10 2023-07-08T12:39:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.720+0000 7f7ec1f21700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f7ebc0cc240 0x7f7ebc0ae440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38620/0 (socket says 172.21.15.12:38620) 2023-07-08T12:39:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.720+0000 7f7ec1f21700 1 -- 172.21.15.12:0/3788245296 learned_addr learned my addr 172.21.15.12:0/3788245296 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.720+0000 7f7ec0f1f700 1 --2- 172.21.15.12:0/3788245296 >> v2:172.21.15.160:3300/0 conn(0x7f7ebc15d390 0x7f7ebc1608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.720+0000 7f7ec1720700 1 --2- 172.21.15.12:0/3788245296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ebc156a10 0x7f7ebc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.720+0000 7f7eb3fff700 1 -- 172.21.15.12:0/3788245296 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337130180 0 0) 0x7f7ebc160410 con 0x7f7ebc0cc240 2023-07-08T12:39:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.721+0000 7f7eb3fff700 1 -- 172.21.15.12:0/3788245296 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea80035b0 con 0x7f7ebc0cc240 2023-07-08T12:39:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.721+0000 7f7ec0f1f700 1 -- 172.21.15.12:0/3788245296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ebc156a10 msgr2=0x7f7ebc06e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.721+0000 7f7ec0f1f700 1 --2- 172.21.15.12:0/3788245296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ebc156a10 0x7f7ebc06e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.721+0000 7f7ec0f1f700 1 -- 172.21.15.12:0/3788245296 >> v1:172.21.15.12:6789/0 conn(0x7f7ebc0cc240 legacy=0x7f7ebc0ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.721+0000 7f7ec0f1f700 1 -- 172.21.15.12:0/3788245296 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc06e6b0 con 0x7f7ebc15d390 2023-07-08T12:39:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.721+0000 7f7ec1720700 1 --2- 172.21.15.12:0/3788245296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ebc156a10 0x7f7ebc06e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:39:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.721+0000 7f7ec0f1f700 1 --2- 172.21.15.12:0/3788245296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ebc15d390 0x7f7ebc1608a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009840 tx=0x7f7eb8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf17e69217e8c11e server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.721+0000 7f7eb3fff700 1 -- 172.21.15.12:0/3788245296 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8011830 con 0x7f7ebc15d390 2023-07-08T12:39:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.721+0000 7f7eb3fff700 1 -- 172.21.15.12:0/3788245296 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb8011990 con 0x7f7ebc15d390 2023-07-08T12:39:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.721+0000 7f7eb3fff700 1 -- 172.21.15.12:0/3788245296 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8011d20 con 0x7f7ebc15d390 2023-07-08T12:39:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.722+0000 7f7ec3984700 1 -- 172.21.15.12:0/3788245296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ebc15d390 msgr2=0x7f7ebc1608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.722+0000 7f7ec3984700 1 --2- 172.21.15.12:0/3788245296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ebc15d390 0x7f7ebc1608a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009840 tx=0x7f7eb8009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.722+0000 7f7ec3984700 1 -- 172.21.15.12:0/3788245296 shutdown_connections 2023-07-08T12:39:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.722+0000 7f7ec3984700 1 --2- 172.21.15.12:0/3788245296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ebc156a10 0x7f7ebc06e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.722+0000 7f7ec3984700 1 --2- 172.21.15.12:0/3788245296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ebc15d390 0x7f7ebc1608a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.722+0000 7f7ec3984700 1 -- 172.21.15.12:0/3788245296 >> 172.21.15.12:0/3788245296 conn(0x7f7ebc098b50 msgr2=0x7f7ebc06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.722+0000 7f7ec3984700 1 -- 172.21.15.12:0/3788245296 shutdown_connections 2023-07-08T12:39:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.722+0000 7f7ec3984700 1 -- 172.21.15.12:0/3788245296 wait complete. 2023-07-08T12:39:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.723+0000 7f7ec3984700 1 Processor -- start 2023-07-08T12:39:15.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.723+0000 7f7ec3984700 1 -- start start 2023-07-08T12:39:15.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec3984700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ebc0cc240 0x7f7ebc155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:15.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec3984700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ebc156a10 0x7f7ebc15bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:15.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec3984700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ebc15d390 0x7f7ebc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:15.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec3984700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0b49b0 con 0x7f7ebc0cc240 2023-07-08T12:39:15.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec3984700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0b2660 con 0x7f7ebc15d390 2023-07-08T12:39:15.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec3984700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0b28b0 con 0x7f7ebc156a10 2023-07-08T12:39:15.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec1720700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ebc0cc240 0x7f7ebc155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:15.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec1720700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ebc0cc240 0x7f7ebc155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51226/0 (socket says 172.21.15.12:51226) 2023-07-08T12:39:15.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec1720700 1 -- 172.21.15.12:0/2424778878 learned_addr learned my addr 172.21.15.12:0/2424778878 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:15.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec1f21700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ebc15d390 0x7f7ebc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:15.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec0f1f700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ebc156a10 0x7f7ebc15bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:15.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec1720700 1 -- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ebc156a10 msgr2=0x7f7ebc15bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec1720700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ebc156a10 0x7f7ebc15bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec1720700 1 -- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ebc15d390 msgr2=0x7f7ebc0b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec1720700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ebc15d390 0x7f7ebc0b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec1720700 1 -- 172.21.15.12:0/2424778878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac009430 con 0x7f7ebc0cc240 2023-07-08T12:39:15.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.724+0000 7f7ec1720700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ebc0cc240 0x7f7ebc155830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00ebe0 tx=0x7f7eac00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:15.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.725+0000 7f7eb1ffb700 1 -- 172.21.15.12:0/2424778878 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac005a20 con 0x7f7ebc0cc240 2023-07-08T12:39:15.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.725+0000 7f7eb1ffb700 1 -- 172.21.15.12:0/2424778878 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac005b80 con 0x7f7ebc0cc240 2023-07-08T12:39:15.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.725+0000 7f7ec3984700 1 -- 172.21.15.12:0/2424778878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb80094a0 con 0x7f7ebc0cc240 2023-07-08T12:39:15.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.725+0000 7f7eb1ffb700 1 -- 172.21.15.12:0/2424778878 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac00c850 con 0x7f7ebc0cc240 2023-07-08T12:39:15.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.725+0000 7f7ec3984700 1 -- 172.21.15.12:0/2424778878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc0b42f0 con 0x7f7ebc0cc240 2023-07-08T12:39:15.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.726+0000 7f7ec3984700 1 -- 172.21.15.12:0/2424778878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc0b4c20 con 0x7f7ebc0cc240 2023-07-08T12:39:15.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.726+0000 7f7eb1ffb700 1 -- 172.21.15.12:0/2424778878 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7eac00c9b0 con 0x7f7ebc0cc240 2023-07-08T12:39:15.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.726+0000 7f7eb1ffb700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7ea006ace0 0x7f7ea006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:15.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.727+0000 7f7eb1ffb700 1 -- 172.21.15.12:0/2424778878 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7eac0899c0 con 0x7f7ebc0cc240 2023-07-08T12:39:15.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.727+0000 7f7ec0f1f700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7ea006ace0 0x7f7ea006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:15.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.727+0000 7f7ec0f1f700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7ea006ace0 0x7f7ea006d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8000c00 tx=0x7f7eb8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:15.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.731+0000 7f7eb1ffb700 1 -- 172.21.15.12:0/2424778878 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7eac054740 con 0x7f7ebc0cc240 2023-07-08T12:39:15.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.926+0000 7f7ec3984700 1 -- 172.21.15.12:0/2424778878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ebc0b4150 con 0x7f7ebc0cc240 2023-07-08T12:39:15.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.927+0000 7f7eb1ffb700 1 -- 172.21.15.12:0/2424778878 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ebc0b4150 con 0x7f7ebc0cc240 2023-07-08T12:39:15.930 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:39:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.929+0000 7f7ec3984700 1 -- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7ea006ace0 msgr2=0x7f7ea006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.929+0000 7f7ec3984700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7ea006ace0 0x7f7ea006d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8000c00 tx=0x7f7eb8008030 comp rx=0 tx=0).stop 2023-07-08T12:39:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.929+0000 7f7ec3984700 1 -- 172.21.15.12:0/2424778878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ebc0cc240 msgr2=0x7f7ebc155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.929+0000 7f7ec3984700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ebc0cc240 0x7f7ebc155830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00ebe0 tx=0x7f7eac00eef0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.929+0000 7f7ec3984700 1 -- 172.21.15.12:0/2424778878 shutdown_connections 2023-07-08T12:39:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.929+0000 7f7ec3984700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f7ea006ace0 0x7f7ea006d190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.929+0000 7f7ec3984700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ebc15d390 0x7f7ebc0b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.929+0000 7f7ec3984700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ebc0cc240 0x7f7ebc155830 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.929+0000 7f7ec3984700 1 --2- 172.21.15.12:0/2424778878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ebc156a10 0x7f7ebc15bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.929+0000 7f7ec3984700 1 -- 172.21.15.12:0/2424778878 >> 172.21.15.12:0/2424778878 conn(0x7f7ebc098b50 msgr2=0x7f7ebc0af850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.929+0000 7f7ec3984700 1 -- 172.21.15.12:0/2424778878 shutdown_connections 2023-07-08T12:39:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:15.929+0000 7f7ec3984700 1 -- 172.21.15.12:0/2424778878 wait complete. 2023-07-08T12:39:15.943 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:39:16.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.069+0000 7fabfcd8b700 1 Processor -- start 2023-07-08T12:39:16.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.069+0000 7fabfcd8b700 1 -- start start 2023-07-08T12:39:16.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.070+0000 7fabfcd8b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabf80aeec0 0x7fabf806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.070+0000 7fabfcd8b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fabf80cdb80 0x7fabf80cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.070+0000 7fabfcd8b700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf815d770 con 0x7fabf806ebe0 2023-07-08T12:39:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.070+0000 7fabfcd8b700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf815a600 con 0x7fabf80cdb80 2023-07-08T12:39:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.070+0000 7fabfcd8b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf815cb60 con 0x7fabf80aeec0 2023-07-08T12:39:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.070+0000 7fabf6d9d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fabf80cdb80 0x7fabf80cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.070+0000 7fabf6d9d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fabf80cdb80 0x7fabf80cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51238/0 (socket says 172.21.15.12:51238) 2023-07-08T12:39:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.070+0000 7fabf6d9d700 1 -- 172.21.15.12:0/815979868 learned_addr learned my addr 172.21.15.12:0/815979868 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.071+0000 7fabf659c700 1 --2- 172.21.15.12:0/815979868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabf80aeec0 0x7fabf806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.071+0000 7fabf6d9d700 1 -- 172.21.15.12:0/815979868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabf80aeec0 msgr2=0x7fabf806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.071+0000 7fabf6d9d700 1 --2- 172.21.15.12:0/815979868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabf80aeec0 0x7fabf806cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.071+0000 7fabf6d9d700 1 -- 172.21.15.12:0/815979868 >> v1:172.21.15.160:6789/0 conn(0x7fabf806ebe0 legacy=0x7fabf80cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.071+0000 7fabf6d9d700 1 -- 172.21.15.12:0/815979868 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf815ce40 con 0x7fabf80cdb80 2023-07-08T12:39:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.071+0000 7fabf659c700 1 --2- 172.21.15.12:0/815979868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabf80aeec0 0x7fabf806cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.071+0000 7fabf6d9d700 1 --2- 172.21.15.12:0/815979868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf80cdb80 0x7fabf80cbe10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fabf000d150 tx=0x7fabf000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45347809da98dcd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.071+0000 7fabf559a700 1 -- 172.21.15.12:0/815979868 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0010070 con 0x7fabf80cdb80 2023-07-08T12:39:16.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.071+0000 7fabf559a700 1 -- 172.21.15.12:0/815979868 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf000ddc0 con 0x7fabf80cdb80 2023-07-08T12:39:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.071+0000 7fabf559a700 1 -- 172.21.15.12:0/815979868 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf000b310 con 0x7fabf80cdb80 2023-07-08T12:39:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.072+0000 7fabfcd8b700 1 -- 172.21.15.12:0/815979868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf80cdb80 msgr2=0x7fabf80cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.072+0000 7fabfcd8b700 1 --2- 172.21.15.12:0/815979868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf80cdb80 0x7fabf80cbe10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fabf000d150 tx=0x7fabf000d460 comp rx=0 tx=0).stop 2023-07-08T12:39:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.072+0000 7fabfcd8b700 1 -- 172.21.15.12:0/815979868 shutdown_connections 2023-07-08T12:39:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.072+0000 7fabfcd8b700 1 --2- 172.21.15.12:0/815979868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf80cdb80 0x7fabf80cbe10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.072+0000 7fabfcd8b700 1 --2- 172.21.15.12:0/815979868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabf80aeec0 0x7fabf806cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.072+0000 7fabfcd8b700 1 -- 172.21.15.12:0/815979868 >> 172.21.15.12:0/815979868 conn(0x7fabf8098b50 msgr2=0x7fabf806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:16.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.072+0000 7fabfcd8b700 1 -- 172.21.15.12:0/815979868 shutdown_connections 2023-07-08T12:39:16.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.072+0000 7fabfcd8b700 1 -- 172.21.15.12:0/815979868 wait complete. 2023-07-08T12:39:16.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.073+0000 7fabfcd8b700 1 Processor -- start 2023-07-08T12:39:16.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.073+0000 7fabfcd8b700 1 -- start start 2023-07-08T12:39:16.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabfcd8b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf806ebe0 0x7fabf814cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabfcd8b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf80aeec0 0x7fabf80b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabfcd8b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabf80cdb80 0x7fabf80c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabfcd8b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf814d270 con 0x7fabf806ebe0 2023-07-08T12:39:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabfcd8b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf806aa70 con 0x7fabf80aeec0 2023-07-08T12:39:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabfcd8b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80cae00 con 0x7fabf80cdb80 2023-07-08T12:39:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabf659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf806ebe0 0x7fabf814cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabf659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf806ebe0 0x7fabf814cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51240/0 (socket says 172.21.15.12:51240) 2023-07-08T12:39:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabf659c700 1 -- 172.21.15.12:0/3288159859 learned_addr learned my addr 172.21.15.12:0/3288159859 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabf6d9d700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabf80cdb80 0x7fabf80c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabf5d9b700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf80aeec0 0x7fabf80b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabf659c700 1 -- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabf80cdb80 msgr2=0x7fabf80c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabf659c700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabf80cdb80 0x7fabf80c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabf659c700 1 -- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf80aeec0 msgr2=0x7fabf80b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabf659c700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf80aeec0 0x7fabf80b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabf659c700 1 -- 172.21.15.12:0/3288159859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf000bd40 con 0x7fabf806ebe0 2023-07-08T12:39:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.074+0000 7fabf5d9b700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf80aeec0 0x7fabf80b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.075+0000 7fabf659c700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf806ebe0 0x7fabf814cd30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fabec00c7f0 tx=0x7fabec00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.075+0000 7fabe6ffd700 1 -- 172.21.15.12:0/3288159859 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec005c20 con 0x7fabf806ebe0 2023-07-08T12:39:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.075+0000 7fabe6ffd700 1 -- 172.21.15.12:0/3288159859 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec005d80 con 0x7fabf806ebe0 2023-07-08T12:39:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.075+0000 7fabfcd8b700 1 -- 172.21.15.12:0/3288159859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf80cb0e0 con 0x7fabf806ebe0 2023-07-08T12:39:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.075+0000 7fabe6ffd700 1 -- 172.21.15.12:0/3288159859 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec011820 con 0x7fabf806ebe0 2023-07-08T12:39:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.075+0000 7fabfcd8b700 1 -- 172.21.15.12:0/3288159859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf80cb6a0 con 0x7fabf806ebe0 2023-07-08T12:39:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.076+0000 7fabfcd8b700 1 -- 172.21.15.12:0/3288159859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf804ea60 con 0x7fabf806ebe0 2023-07-08T12:39:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.076+0000 7fabe6ffd700 1 -- 172.21.15.12:0/3288159859 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fabec011980 con 0x7fabf806ebe0 2023-07-08T12:39:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.077+0000 7fabe6ffd700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabd806ace0 0x7fabd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.077+0000 7fabe6ffd700 1 -- 172.21.15.12:0/3288159859 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fabec089b70 con 0x7fabf806ebe0 2023-07-08T12:39:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.077+0000 7fabf5d9b700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabd806ace0 0x7fabd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.077+0000 7fabf5d9b700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabd806ace0 0x7fabd806d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fabf80adfc0 tx=0x7fabe8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.081+0000 7fabe6ffd700 1 -- 172.21.15.12:0/3288159859 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fabec054850 con 0x7fabf806ebe0 2023-07-08T12:39:16.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.278+0000 7fabfcd8b700 1 -- 172.21.15.12:0/3288159859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fabf8066af0 con 0x7fabf806ebe0 2023-07-08T12:39:16.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.278+0000 7fabe6ffd700 1 -- 172.21.15.12:0/3288159859 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fabec057e70 con 0x7fabf806ebe0 2023-07-08T12:39:16.281 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:39:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.280+0000 7fabfcd8b700 1 -- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabd806ace0 msgr2=0x7fabd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.280+0000 7fabfcd8b700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabd806ace0 0x7fabd806d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fabf80adfc0 tx=0x7fabe8009040 comp rx=0 tx=0).stop 2023-07-08T12:39:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.280+0000 7fabfcd8b700 1 -- 172.21.15.12:0/3288159859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf806ebe0 msgr2=0x7fabf814cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.280+0000 7fabfcd8b700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf806ebe0 0x7fabf814cd30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fabec00c7f0 tx=0x7fabec00cbb0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.280+0000 7fabfcd8b700 1 -- 172.21.15.12:0/3288159859 shutdown_connections 2023-07-08T12:39:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.280+0000 7fabfcd8b700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fabd806ace0 0x7fabd806d190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.280+0000 7fabfcd8b700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabf80cdb80 0x7fabf80c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.280+0000 7fabfcd8b700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf806ebe0 0x7fabf814cd30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.280+0000 7fabfcd8b700 1 --2- 172.21.15.12:0/3288159859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabf80aeec0 0x7fabf80b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.280+0000 7fabfcd8b700 1 -- 172.21.15.12:0/3288159859 >> 172.21.15.12:0/3288159859 conn(0x7fabf8098b50 msgr2=0x7fabf80cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:16.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.280+0000 7fabfcd8b700 1 -- 172.21.15.12:0/3288159859 shutdown_connections 2023-07-08T12:39:16.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.280+0000 7fabfcd8b700 1 -- 172.21.15.12:0/3288159859 wait complete. 2023-07-08T12:39:16.294 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-08T12:39:16.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.419+0000 7fd6fec21700 1 Processor -- start 2023-07-08T12:39:16.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.419+0000 7fd6fec21700 1 -- start start 2023-07-08T12:39:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.420+0000 7fd6fec21700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd6f80cc240 0x7fd6f80cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.420+0000 7fd6fec21700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f80ccbe0 con 0x7fd6f806e0c0 2023-07-08T12:39:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.420+0000 7fd6fec21700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f8160410 con 0x7fd6f8156a10 2023-07-08T12:39:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.420+0000 7fd6fec21700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f8156de0 con 0x7fd6f80cc240 2023-07-08T12:39:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.420+0000 7fd6f7fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd6f806e0c0 0x7fd6f81608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38646/0 (socket says 172.21.15.12:38646) 2023-07-08T12:39:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.420+0000 7fd6f7fff700 1 -- 172.21.15.12:0/3565538568 learned_addr learned my addr 172.21.15.12:0/3565538568 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6fd1be700 1 --2- 172.21.15.12:0/3565538568 >> v2:172.21.15.160:3300/0 conn(0x7fd6f80cc240 0x7fd6f80cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056885274 0 0) 0x7fd6f80ccbe0 con 0x7fd6f806e0c0 2023-07-08T12:39:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6e80035b0 con 0x7fd6f806e0c0 2023-07-08T12:39:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4268429029 0 0) 0x7fd6e80035b0 con 0x7fd6f806e0c0 2023-07-08T12:39:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 >> v2:172.21.15.160:3300/0 conn(0x7fd6f80cc240 msgr2=0x7fd6f80cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 --2- 172.21.15.12:0/3565538568 >> v2:172.21.15.160:3300/0 conn(0x7fd6f80cc240 0x7fd6f80cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 >> v1:172.21.15.160:6789/0 conn(0x7fd6f8156a10 legacy=0x7fd6f815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f806ec70 con 0x7fd6f806e0c0 2023-07-08T12:39:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7fd6e4001670 con 0x7fd6f806e0c0 2023-07-08T12:39:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 >> v1:172.21.15.12:6789/0 conn(0x7fd6f806e0c0 legacy=0x7fd6f81608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 --2- 172.21.15.12:0/3565538568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8003a00 0x7fd6e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 --2- 172.21.15.12:0/3565538568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6e8004850 0x7fd6e8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 --2- 172.21.15.12:0/3565538568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6e8005200 0x7fd6e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8156de0 con 0x7fd6e8003a00 2023-07-08T12:39:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e800c080 con 0x7fd6e8005200 2023-07-08T12:39:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e800c240 con 0x7fd6e8004850 2023-07-08T12:39:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6fc9bd700 1 --2- 172.21.15.12:0/3565538568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8003a00 0x7fd6e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6fd1be700 1 --2- 172.21.15.12:0/3565538568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6e8005200 0x7fd6e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.421+0000 7fd6f7fff700 1 --2- 172.21.15.12:0/3565538568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6e8004850 0x7fd6e8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fc9bd700 1 -- 172.21.15.12:0/3565538568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6e8004850 msgr2=0x7fd6e8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fc9bd700 1 --2- 172.21.15.12:0/3565538568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6e8004850 0x7fd6e8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fc9bd700 1 -- 172.21.15.12:0/3565538568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6e8005200 msgr2=0x7fd6e800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fc9bd700 1 --2- 172.21.15.12:0/3565538568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6e8005200 0x7fd6e800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fc9bd700 1 -- 172.21.15.12:0/3565538568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e80035b0 con 0x7fd6e8003a00 2023-07-08T12:39:16.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fd1be700 1 -- 172.21.15.12:0/3565538568 reap_dead start 2023-07-08T12:39:16.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6f7fff700 1 --2- 172.21.15.12:0/3565538568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6e8004850 0x7fd6e8004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:16.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fc9bd700 1 --2- 172.21.15.12:0/3565538568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8003a00 0x7fd6e8003e10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec001820 tx=0x7fd6e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c4010b253454cf server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:16.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec0012e0 con 0x7fd6e8003a00 2023-07-08T12:39:16.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ec0052d0 con 0x7fd6e8003a00 2023-07-08T12:39:16.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6f77fe700 1 -- 172.21.15.12:0/3565538568 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec0036d0 con 0x7fd6e8003a00 2023-07-08T12:39:16.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fec21700 1 -- 172.21.15.12:0/3565538568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8003a00 msgr2=0x7fd6e8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fec21700 1 --2- 172.21.15.12:0/3565538568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8003a00 0x7fd6e8003e10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec001820 tx=0x7fd6e8005880 comp rx=0 tx=0).stop 2023-07-08T12:39:16.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fec21700 1 -- 172.21.15.12:0/3565538568 shutdown_connections 2023-07-08T12:39:16.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fec21700 1 --2- 172.21.15.12:0/3565538568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6e8003a00 0x7fd6e8003e10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fec21700 1 -- 172.21.15.12:0/3565538568 >> 172.21.15.12:0/3565538568 conn(0x7fd6f8098b50 msgr2=0x7fd6f806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:16.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fec21700 1 -- 172.21.15.12:0/3565538568 shutdown_connections 2023-07-08T12:39:16.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.422+0000 7fd6fec21700 1 -- 172.21.15.12:0/3565538568 wait complete. 2023-07-08T12:39:16.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.424+0000 7fd6fec21700 1 Processor -- start 2023-07-08T12:39:16.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.424+0000 7fd6fec21700 1 -- start start 2023-07-08T12:39:16.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6fec21700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6e8003a00 0x7fd6f81522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6fec21700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f80b4fb0 0x7fd6f80ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6fec21700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6f80b5420 0x7fd6f80ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6fec21700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f81543c0 con 0x7fd6f80b4fb0 2023-07-08T12:39:16.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6fec21700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80b4760 con 0x7fd6f80b5420 2023-07-08T12:39:16.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6fec21700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80b36c0 con 0x7fd6e8003a00 2023-07-08T12:39:16.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6f7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f80b4fb0 0x7fd6f80ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6f7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f80b4fb0 0x7fd6f80ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51262/0 (socket says 172.21.15.12:51262) 2023-07-08T12:39:16.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6f7fff700 1 -- 172.21.15.12:0/2494517350 learned_addr learned my addr 172.21.15.12:0/2494517350 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:16.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6fd1be700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6f80b5420 0x7fd6f80ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6fc9bd700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6e8003a00 0x7fd6f81522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6f7fff700 1 -- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6e8003a00 msgr2=0x7fd6f81522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6f7fff700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6e8003a00 0x7fd6f81522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6f7fff700 1 -- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6f80b5420 msgr2=0x7fd6f80ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6f7fff700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6f80b5420 0x7fd6f80ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6f7fff700 1 -- 172.21.15.12:0/2494517350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f80b39a0 con 0x7fd6f80b4fb0 2023-07-08T12:39:16.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6fc9bd700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6e8003a00 0x7fd6f81522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:16.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6fd1be700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6f80b5420 0x7fd6f80ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:16.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6f7fff700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f80b4fb0 0x7fd6f80ce330 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4001c90 tx=0x7fd6e400eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:16.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6f57fa700 1 -- 172.21.15.12:0/2494517350 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4012590 con 0x7fd6f80b4fb0 2023-07-08T12:39:16.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6f57fa700 1 -- 172.21.15.12:0/2494517350 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e40126f0 con 0x7fd6f80b4fb0 2023-07-08T12:39:16.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6fec21700 1 -- 172.21.15.12:0/2494517350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f80b3c80 con 0x7fd6f80b4fb0 2023-07-08T12:39:16.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6f57fa700 1 -- 172.21.15.12:0/2494517350 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4012a60 con 0x7fd6f80b4fb0 2023-07-08T12:39:16.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.425+0000 7fd6fec21700 1 -- 172.21.15.12:0/2494517350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f80b4240 con 0x7fd6f80b4fb0 2023-07-08T12:39:16.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.426+0000 7fd6fec21700 1 -- 172.21.15.12:0/2494517350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f80b4460 con 0x7fd6f80b4fb0 2023-07-08T12:39:16.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.427+0000 7fd6f57fa700 1 -- 172.21.15.12:0/2494517350 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd6e4012bc0 con 0x7fd6f80b4fb0 2023-07-08T12:39:16.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.427+0000 7fd6f57fa700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd6dc06ace0 0x7fd6dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.427+0000 7fd6f57fa700 1 -- 172.21.15.12:0/2494517350 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd6e40893d0 con 0x7fd6f80b4fb0 2023-07-08T12:39:16.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.427+0000 7fd6fc9bd700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd6dc06ace0 0x7fd6dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.428+0000 7fd6fc9bd700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd6dc06ace0 0x7fd6dc06d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec015040 tx=0x7fd6ec011000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:16.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.431+0000 7fd6f57fa700 1 -- 172.21.15.12:0/2494517350 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd6e4057c70 con 0x7fd6f80b4fb0 2023-07-08T12:39:16.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.634+0000 7fd6fec21700 1 -- 172.21.15.12:0/2494517350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd6f804ea60 con 0x7fd6f80b4fb0 2023-07-08T12:39:16.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.634+0000 7fd6f57fa700 1 -- 172.21.15.12:0/2494517350 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd6e4057800 con 0x7fd6f80b4fb0 2023-07-08T12:39:16.637 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:16.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.637+0000 7fd6fec21700 1 -- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd6dc06ace0 msgr2=0x7fd6dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.637+0000 7fd6fec21700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd6dc06ace0 0x7fd6dc06d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec015040 tx=0x7fd6ec011000 comp rx=0 tx=0).stop 2023-07-08T12:39:16.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.637+0000 7fd6fec21700 1 -- 172.21.15.12:0/2494517350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f80b4fb0 msgr2=0x7fd6f80ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.637+0000 7fd6fec21700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f80b4fb0 0x7fd6f80ce330 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4001c90 tx=0x7fd6e400eb10 comp rx=0 tx=0).stop 2023-07-08T12:39:16.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.637+0000 7fd6fec21700 1 -- 172.21.15.12:0/2494517350 shutdown_connections 2023-07-08T12:39:16.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.637+0000 7fd6fec21700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fd6dc06ace0 0x7fd6dc06d190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.637+0000 7fd6fec21700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6f80b5420 0x7fd6f80ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.637+0000 7fd6fec21700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6e8003a00 0x7fd6f81522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.637+0000 7fd6fec21700 1 --2- 172.21.15.12:0/2494517350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6f80b4fb0 0x7fd6f80ce330 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.637+0000 7fd6fec21700 1 -- 172.21.15.12:0/2494517350 >> 172.21.15.12:0/2494517350 conn(0x7fd6f8098b50 msgr2=0x7fd6f8155d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:16.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.637+0000 7fd6fec21700 1 -- 172.21.15.12:0/2494517350 shutdown_connections 2023-07-08T12:39:16.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.637+0000 7fd6fec21700 1 -- 172.21.15.12:0/2494517350 wait complete. 2023-07-08T12:39:16.652 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-08T12:39:16.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.778+0000 7fe00b978700 1 Processor -- start 2023-07-08T12:39:16.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.778+0000 7fe00b978700 1 -- start start 2023-07-08T12:39:16.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.779+0000 7fe00b978700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe004156a10 0x7fe00406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.779+0000 7fe00b978700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe00415d390 0x7fe0041608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.779+0000 7fe00b978700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe004160410 con 0x7fe0040cc240 2023-07-08T12:39:16.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.779+0000 7fe00b978700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe004156de0 con 0x7fe00415d390 2023-07-08T12:39:16.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.779+0000 7fe00b978700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00406ec70 con 0x7fe004156a10 2023-07-08T12:39:16.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.779+0000 7fe008f13700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe00415d390 0x7fe0041608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.779+0000 7fe008f13700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe00415d390 0x7fe0041608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51270/0 (socket says 172.21.15.12:51270) 2023-07-08T12:39:16.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.779+0000 7fe008f13700 1 -- 172.21.15.12:0/964845314 learned_addr learned my addr 172.21.15.12:0/964845314 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:16.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.779+0000 7fe009714700 1 --2- 172.21.15.12:0/964845314 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe004156a10 0x7fe00406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.779+0000 7fe008f13700 1 -- 172.21.15.12:0/964845314 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe004156a10 msgr2=0x7fe00406e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:16.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.779+0000 7fe008f13700 1 --2- 172.21.15.12:0/964845314 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe004156a10 0x7fe00406e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.779+0000 7fe008f13700 1 -- 172.21.15.12:0/964845314 >> v1:172.21.15.160:6789/0 conn(0x7fe0040cc240 legacy=0x7fe0040ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.779+0000 7fe008f13700 1 -- 172.21.15.12:0/964845314 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00406e6b0 con 0x7fe00415d390 2023-07-08T12:39:16.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.780+0000 7fe009714700 1 --2- 172.21.15.12:0/964845314 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe004156a10 0x7fe00406e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:16.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.780+0000 7fe008f13700 1 --2- 172.21.15.12:0/964845314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe00415d390 0x7fe0041608a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe0000098f0 tx=0x7fe000009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a1db0f21e7731 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:16.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.780+0000 7fdffbfff700 1 -- 172.21.15.12:0/964845314 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00000b8a0 con 0x7fe00415d390 2023-07-08T12:39:16.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.780+0000 7fdffbfff700 1 -- 172.21.15.12:0/964845314 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00000ba00 con 0x7fe00415d390 2023-07-08T12:39:16.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.780+0000 7fdffbfff700 1 -- 172.21.15.12:0/964845314 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00000bd90 con 0x7fe00415d390 2023-07-08T12:39:16.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.780+0000 7fe00b978700 1 -- 172.21.15.12:0/964845314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe00415d390 msgr2=0x7fe0041608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.780+0000 7fe00b978700 1 --2- 172.21.15.12:0/964845314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe00415d390 0x7fe0041608a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe0000098f0 tx=0x7fe000009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:16.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.780+0000 7fe00b978700 1 -- 172.21.15.12:0/964845314 shutdown_connections 2023-07-08T12:39:16.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.780+0000 7fe00b978700 1 --2- 172.21.15.12:0/964845314 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe004156a10 0x7fe00406e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.780+0000 7fe00b978700 1 --2- 172.21.15.12:0/964845314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe00415d390 0x7fe0041608a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.780+0000 7fe00b978700 1 -- 172.21.15.12:0/964845314 >> 172.21.15.12:0/964845314 conn(0x7fe004098b50 msgr2=0x7fe00406b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:16.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.781+0000 7fe00b978700 1 -- 172.21.15.12:0/964845314 shutdown_connections 2023-07-08T12:39:16.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.781+0000 7fe00b978700 1 -- 172.21.15.12:0/964845314 wait complete. 2023-07-08T12:39:16.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.782+0000 7fe00b978700 1 Processor -- start 2023-07-08T12:39:16.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.782+0000 7fe00b978700 1 -- start start 2023-07-08T12:39:16.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe00b978700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0040cc240 0x7fe00415bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe00b978700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004156a10 0x7fe00415c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe00b978700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe00415d390 0x7fe0041552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe00b978700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00415af10 con 0x7fe004156a10 2023-07-08T12:39:16.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe00b978700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00415ca40 con 0x7fe00415d390 2023-07-08T12:39:16.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe00b978700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0041560c0 con 0x7fe0040cc240 2023-07-08T12:39:16.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe008f13700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004156a10 0x7fe00415c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe008f13700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004156a10 0x7fe00415c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51276/0 (socket says 172.21.15.12:51276) 2023-07-08T12:39:16.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe008f13700 1 -- 172.21.15.12:0/185804673 learned_addr learned my addr 172.21.15.12:0/185804673 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:16.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe009f15700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe00415d390 0x7fe0041552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe009714700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0040cc240 0x7fe00415bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe008f13700 1 -- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0040cc240 msgr2=0x7fe00415bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe008f13700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0040cc240 0x7fe00415bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe008f13700 1 -- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe00415d390 msgr2=0x7fe0041552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe008f13700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe00415d390 0x7fe0041552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe008f13700 1 -- 172.21.15.12:0/185804673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe000009550 con 0x7fe004156a10 2023-07-08T12:39:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe009f15700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe00415d390 0x7fe0041552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe008f13700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004156a10 0x7fe00415c500 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe000000c00 tx=0x7fe000004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fdff9ffb700 1 -- 172.21.15.12:0/185804673 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000004480 con 0x7fe004156a10 2023-07-08T12:39:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fdff9ffb700 1 -- 172.21.15.12:0/185804673 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00001f070 con 0x7fe004156a10 2023-07-08T12:39:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fe00b978700 1 -- 172.21.15.12:0/185804673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00415b190 con 0x7fe004156a10 2023-07-08T12:39:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.783+0000 7fdff9ffb700 1 -- 172.21.15.12:0/185804673 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0000038c0 con 0x7fe004156a10 2023-07-08T12:39:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.784+0000 7fe00b978700 1 -- 172.21.15.12:0/185804673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00415b750 con 0x7fe004156a10 2023-07-08T12:39:16.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.784+0000 7fe00b978700 1 -- 172.21.15.12:0/185804673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00404ea60 con 0x7fe004156a10 2023-07-08T12:39:16.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.785+0000 7fdff9ffb700 1 -- 172.21.15.12:0/185804673 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe000021030 con 0x7fe004156a10 2023-07-08T12:39:16.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.785+0000 7fdff9ffb700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdfe806ace0 0x7fdfe806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:16.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.785+0000 7fdff9ffb700 1 -- 172.21.15.12:0/185804673 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe00008acd0 con 0x7fe004156a10 2023-07-08T12:39:16.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.786+0000 7fe009714700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdfe806ace0 0x7fdfe806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:16.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.786+0000 7fe009714700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdfe806ace0 0x7fdfe806d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdff400c920 tx=0x7fdff4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:16.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.790+0000 7fdff9ffb700 1 -- 172.21.15.12:0/185804673 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe000055c00 con 0x7fe004156a10 2023-07-08T12:39:16.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.986+0000 7fe00b978700 1 -- 172.21.15.12:0/185804673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0040ccde0 con 0x7fe004156a10 2023-07-08T12:39:16.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.986+0000 7fdff9ffb700 1 -- 172.21.15.12:0/185804673 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe000059220 con 0x7fe004156a10 2023-07-08T12:39:16.989 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-07-08T12:39:16.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.988+0000 7fe00b978700 1 -- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdfe806ace0 msgr2=0x7fdfe806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.988+0000 7fe00b978700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdfe806ace0 0x7fdfe806d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdff400c920 tx=0x7fdff4008040 comp rx=0 tx=0).stop 2023-07-08T12:39:16.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.988+0000 7fe00b978700 1 -- 172.21.15.12:0/185804673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004156a10 msgr2=0x7fe00415c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:16.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.988+0000 7fe00b978700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004156a10 0x7fe00415c500 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe000000c00 tx=0x7fe000004750 comp rx=0 tx=0).stop 2023-07-08T12:39:16.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.989+0000 7fe00b978700 1 -- 172.21.15.12:0/185804673 shutdown_connections 2023-07-08T12:39:16.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.989+0000 7fe00b978700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fdfe806ace0 0x7fdfe806d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.989+0000 7fe00b978700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe00415d390 0x7fe0041552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.989+0000 7fe00b978700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0040cc240 0x7fe00415bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.989+0000 7fe00b978700 1 --2- 172.21.15.12:0/185804673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe004156a10 0x7fe00415c500 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:16.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.989+0000 7fe00b978700 1 -- 172.21.15.12:0/185804673 >> 172.21.15.12:0/185804673 conn(0x7fe004098b50 msgr2=0x7fe0040af680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:16.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.989+0000 7fe00b978700 1 -- 172.21.15.12:0/185804673 shutdown_connections 2023-07-08T12:39:16.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:16.989+0000 7fe00b978700 1 -- 172.21.15.12:0/185804673 wait complete. 2023-07-08T12:39:17.003 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-08T12:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.129+0000 7f0e41a7b700 1 Processor -- start 2023-07-08T12:39:17.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.129+0000 7f0e41a7b700 1 -- start start 2023-07-08T12:39:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.130+0000 7f0e41a7b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0e3c156a10 0x7f0e3c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.130+0000 7f0e41a7b700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0e3c15d390 0x7f0e3c1608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.130+0000 7f0e41a7b700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3c160410 con 0x7f0e3c0cc240 2023-07-08T12:39:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.130+0000 7f0e41a7b700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3c156de0 con 0x7f0e3c156a10 2023-07-08T12:39:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.130+0000 7f0e41a7b700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3c06ec70 con 0x7f0e3c15d390 2023-07-08T12:39:17.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.130+0000 7f0e3b7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0e3c156a10 0x7f0e3c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.130+0000 7f0e3b7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0e3c156a10 0x7f0e3c06e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51282/0 (socket says 172.21.15.12:51282) 2023-07-08T12:39:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.130+0000 7f0e3b7fe700 1 -- 172.21.15.12:0/2768361668 learned_addr learned my addr 172.21.15.12:0/2768361668 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.130+0000 7f0e3affd700 1 --2- 172.21.15.12:0/2768361668 >> v2:172.21.15.160:3300/0 conn(0x7f0e3c15d390 0x7f0e3c1608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.130+0000 7f0e3b7fe700 1 -- 172.21.15.12:0/2768361668 >> v2:172.21.15.160:3300/0 conn(0x7f0e3c15d390 msgr2=0x7f0e3c1608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.130+0000 7f0e3b7fe700 1 --2- 172.21.15.12:0/2768361668 >> v2:172.21.15.160:3300/0 conn(0x7f0e3c15d390 0x7f0e3c1608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.130+0000 7f0e3b7fe700 1 -- 172.21.15.12:0/2768361668 >> v1:172.21.15.160:6789/0 conn(0x7f0e3c0cc240 legacy=0x7f0e3c0ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:17.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e3b7fe700 1 -- 172.21.15.12:0/2768361668 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c06e6b0 con 0x7f0e3c156a10 2023-07-08T12:39:17.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e3affd700 1 --2- 172.21.15.12:0/2768361668 >> v2:172.21.15.160:3300/0 conn(0x7f0e3c15d390 0x7f0e3c1608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:17.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e3b7fe700 1 --2- 172.21.15.12:0/2768361668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e3c156a10 0x7f0e3c06e0c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c072690 tx=0x7f0e2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a645260b752824 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:17.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e3a7fc700 1 -- 172.21.15.12:0/2768361668 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c005e30 con 0x7f0e3c156a10 2023-07-08T12:39:17.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e3a7fc700 1 -- 172.21.15.12:0/2768361668 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c004030 con 0x7f0e3c156a10 2023-07-08T12:39:17.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e3a7fc700 1 -- 172.21.15.12:0/2768361668 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0043e0 con 0x7f0e3c156a10 2023-07-08T12:39:17.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e41a7b700 1 -- 172.21.15.12:0/2768361668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e3c156a10 msgr2=0x7f0e3c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e41a7b700 1 --2- 172.21.15.12:0/2768361668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e3c156a10 0x7f0e3c06e0c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c072690 tx=0x7f0e2c009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e41a7b700 1 -- 172.21.15.12:0/2768361668 shutdown_connections 2023-07-08T12:39:17.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e41a7b700 1 --2- 172.21.15.12:0/2768361668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e3c156a10 0x7f0e3c06e0c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e41a7b700 1 --2- 172.21.15.12:0/2768361668 >> v2:172.21.15.160:3300/0 conn(0x7f0e3c15d390 0x7f0e3c1608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e41a7b700 1 -- 172.21.15.12:0/2768361668 >> 172.21.15.12:0/2768361668 conn(0x7f0e3c098b50 msgr2=0x7f0e3c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:17.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e41a7b700 1 -- 172.21.15.12:0/2768361668 shutdown_connections 2023-07-08T12:39:17.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.131+0000 7f0e41a7b700 1 -- 172.21.15.12:0/2768361668 wait complete. 2023-07-08T12:39:17.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.133+0000 7f0e41a7b700 1 Processor -- start 2023-07-08T12:39:17.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.133+0000 7f0e41a7b700 1 -- start start 2023-07-08T12:39:17.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.134+0000 7f0e41a7b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c0cc240 0x7f0e3c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.134+0000 7f0e41a7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e3c156a10 0x7f0e3c0b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.134+0000 7f0e41a7b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c15d390 0x7f0e3c1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.134+0000 7f0e41a7b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c1e6fb0 con 0x7f0e3c156a10 2023-07-08T12:39:17.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.134+0000 7f0e41a7b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c1f6bd0 con 0x7f0e3c15d390 2023-07-08T12:39:17.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.134+0000 7f0e41a7b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c1e71c0 con 0x7f0e3c0cc240 2023-07-08T12:39:17.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.134+0000 7f0e3affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e3c156a10 0x7f0e3c0b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.134+0000 7f0e3affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e3c156a10 0x7f0e3c0b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51290/0 (socket says 172.21.15.12:51290) 2023-07-08T12:39:17.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.134+0000 7f0e3affd700 1 -- 172.21.15.12:0/1219529494 learned_addr learned my addr 172.21.15.12:0/1219529494 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:17.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.134+0000 7f0e3b7fe700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c0cc240 0x7f0e3c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.134+0000 7f0e3bfff700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c15d390 0x7f0e3c1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.135+0000 7f0e3affd700 1 -- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c0cc240 msgr2=0x7f0e3c0b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.135+0000 7f0e3affd700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c0cc240 0x7f0e3c0b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.135+0000 7f0e3affd700 1 -- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c15d390 msgr2=0x7f0e3c1552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.135+0000 7f0e3affd700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c15d390 0x7f0e3c1552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.135+0000 7f0e3affd700 1 -- 172.21.15.12:0/1219529494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c009500 con 0x7f0e3c156a10 2023-07-08T12:39:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.135+0000 7f0e3affd700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e3c156a10 0x7f0e3c0b2660 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400e810 tx=0x7f0e2400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.135+0000 7f0e33fff700 1 -- 172.21.15.12:0/1219529494 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2400c070 con 0x7f0e3c156a10 2023-07-08T12:39:17.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.135+0000 7f0e33fff700 1 -- 172.21.15.12:0/1219529494 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e24004230 con 0x7f0e3c156a10 2023-07-08T12:39:17.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.135+0000 7f0e41a7b700 1 -- 172.21.15.12:0/1219529494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c1e74a0 con 0x7f0e3c156a10 2023-07-08T12:39:17.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.135+0000 7f0e33fff700 1 -- 172.21.15.12:0/1219529494 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e240036a0 con 0x7f0e3c156a10 2023-07-08T12:39:17.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.135+0000 7f0e41a7b700 1 -- 172.21.15.12:0/1219529494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c0b5370 con 0x7f0e3c156a10 2023-07-08T12:39:17.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.136+0000 7f0e41a7b700 1 -- 172.21.15.12:0/1219529494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3c04ea60 con 0x7f0e3c156a10 2023-07-08T12:39:17.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.137+0000 7f0e33fff700 1 -- 172.21.15.12:0/1219529494 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e24003800 con 0x7f0e3c156a10 2023-07-08T12:39:17.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.137+0000 7f0e33fff700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0e1c06ace0 0x7f0e1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.137+0000 7f0e33fff700 1 -- 172.21.15.12:0/1219529494 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0e240893d0 con 0x7f0e3c156a10 2023-07-08T12:39:17.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.137+0000 7f0e3b7fe700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0e1c06ace0 0x7f0e1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.138+0000 7f0e3b7fe700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0e1c06ace0 0x7f0e1c06d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c072690 tx=0x7f0e2c008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:17.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.141+0000 7f0e33fff700 1 -- 172.21.15.12:0/1219529494 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0e240541e0 con 0x7f0e3c156a10 2023-07-08T12:39:17.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.340+0000 7f0e41a7b700 1 -- 172.21.15.12:0/1219529494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0e3c0cbc80 con 0x7f0e3c156a10 2023-07-08T12:39:17.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.341+0000 7f0e33fff700 1 -- 172.21.15.12:0/1219529494 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0e24057800 con 0x7f0e3c156a10 2023-07-08T12:39:17.344 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-07-08T12:39:17.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.343+0000 7f0e41a7b700 1 -- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0e1c06ace0 msgr2=0x7f0e1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.343+0000 7f0e41a7b700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0e1c06ace0 0x7f0e1c06d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c072690 tx=0x7f0e2c008070 comp rx=0 tx=0).stop 2023-07-08T12:39:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.343+0000 7f0e41a7b700 1 -- 172.21.15.12:0/1219529494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e3c156a10 msgr2=0x7f0e3c0b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.343+0000 7f0e41a7b700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e3c156a10 0x7f0e3c0b2660 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400e810 tx=0x7f0e2400eb20 comp rx=0 tx=0).stop 2023-07-08T12:39:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.343+0000 7f0e41a7b700 1 -- 172.21.15.12:0/1219529494 shutdown_connections 2023-07-08T12:39:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.343+0000 7f0e41a7b700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0e1c06ace0 0x7f0e1c06d190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.343+0000 7f0e41a7b700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c15d390 0x7f0e3c1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.343+0000 7f0e41a7b700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c0cc240 0x7f0e3c0b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.343+0000 7f0e41a7b700 1 --2- 172.21.15.12:0/1219529494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e3c156a10 0x7f0e3c0b2660 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.343+0000 7f0e41a7b700 1 -- 172.21.15.12:0/1219529494 >> 172.21.15.12:0/1219529494 conn(0x7f0e3c098b50 msgr2=0x7f0e3c0ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.343+0000 7f0e41a7b700 1 -- 172.21.15.12:0/1219529494 shutdown_connections 2023-07-08T12:39:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.343+0000 7f0e41a7b700 1 -- 172.21.15.12:0/1219529494 wait complete. 2023-07-08T12:39:17.358 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-08T12:39:17.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.481+0000 7fb678468700 1 Processor -- start 2023-07-08T12:39:17.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.481+0000 7fb678468700 1 -- start start 2023-07-08T12:39:17.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb678468700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6700b1c70 0x7fb6700ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb678468700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb67015d6c0 0x7fb670071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb678468700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67015d230 con 0x7fb6700ce1b0 2023-07-08T12:39:17.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb678468700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6700c9e60 con 0x7fb67015d6c0 2023-07-08T12:39:17.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb678468700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700ceac0 con 0x7fb6700b1c70 2023-07-08T12:39:17.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb676204700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb6700ce1b0 0x7fb6700cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38692/0 (socket says 172.21.15.12:38692) 2023-07-08T12:39:17.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb676204700 1 -- 172.21.15.12:0/884244816 learned_addr learned my addr 172.21.15.12:0/884244816 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:17.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb676a05700 1 --2- 172.21.15.12:0/884244816 >> v2:172.21.15.160:3300/0 conn(0x7fb67015d6c0 0x7fb670071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb675a03700 1 --2- 172.21.15.12:0/884244816 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6700b1c70 0x7fb6700ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb675202700 1 -- 172.21.15.12:0/884244816 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075866425 0 0) 0x7fb67015d230 con 0x7fb6700ce1b0 2023-07-08T12:39:17.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb675202700 1 -- 172.21.15.12:0/884244816 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb65c0057c0 con 0x7fb6700ce1b0 2023-07-08T12:39:17.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb675202700 1 -- 172.21.15.12:0/884244816 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 532402615 0 0) 0x7fb65c0057c0 con 0x7fb6700ce1b0 2023-07-08T12:39:17.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb675a03700 1 -- 172.21.15.12:0/884244816 >> v2:172.21.15.160:3300/0 conn(0x7fb67015d6c0 msgr2=0x7fb670071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:17.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb675a03700 1 --2- 172.21.15.12:0/884244816 >> v2:172.21.15.160:3300/0 conn(0x7fb67015d6c0 0x7fb670071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.482+0000 7fb675a03700 1 -- 172.21.15.12:0/884244816 >> v1:172.21.15.12:6789/0 conn(0x7fb6700ce1b0 legacy=0x7fb6700cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.483+0000 7fb675a03700 1 -- 172.21.15.12:0/884244816 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6700cdbc0 con 0x7fb6700b1c70 2023-07-08T12:39:17.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.483+0000 7fb676a05700 1 --2- 172.21.15.12:0/884244816 >> v2:172.21.15.160:3300/0 conn(0x7fb67015d6c0 0x7fb670071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:39:17.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.483+0000 7fb675a03700 1 --2- 172.21.15.12:0/884244816 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6700b1c70 0x7fb6700ce580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb660009840 tx=0x7fb660009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f44f467b2ffedbb server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:17.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.483+0000 7fb675202700 1 -- 172.21.15.12:0/884244816 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c001670 con 0x7fb6700b1c70 2023-07-08T12:39:17.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.483+0000 7fb675202700 1 -- 172.21.15.12:0/884244816 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb660004030 con 0x7fb6700b1c70 2023-07-08T12:39:17.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.483+0000 7fb675202700 1 -- 172.21.15.12:0/884244816 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6600043c0 con 0x7fb6700b1c70 2023-07-08T12:39:17.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.484+0000 7fb678468700 1 -- 172.21.15.12:0/884244816 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6700b1c70 msgr2=0x7fb6700ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.484+0000 7fb678468700 1 --2- 172.21.15.12:0/884244816 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6700b1c70 0x7fb6700ce580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb660009840 tx=0x7fb660009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:17.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.484+0000 7fb678468700 1 -- 172.21.15.12:0/884244816 shutdown_connections 2023-07-08T12:39:17.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.484+0000 7fb678468700 1 --2- 172.21.15.12:0/884244816 >> v2:172.21.15.160:3300/0 conn(0x7fb67015d6c0 0x7fb670071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.484+0000 7fb678468700 1 --2- 172.21.15.12:0/884244816 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6700b1c70 0x7fb6700ce580 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.484+0000 7fb678468700 1 -- 172.21.15.12:0/884244816 >> 172.21.15.12:0/884244816 conn(0x7fb6700cc290 msgr2=0x7fb6700cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:17.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.484+0000 7fb678468700 1 -- 172.21.15.12:0/884244816 shutdown_connections 2023-07-08T12:39:17.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.484+0000 7fb678468700 1 -- 172.21.15.12:0/884244816 wait complete. 2023-07-08T12:39:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.485+0000 7fb678468700 1 Processor -- start 2023-07-08T12:39:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.485+0000 7fb678468700 1 -- start start 2023-07-08T12:39:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb678468700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6700b1c70 0x7fb67014ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb678468700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6700ce1b0 0x7fb6700adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb678468700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb67015d6c0 0x7fb6701f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb678468700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67014f370 con 0x7fb67015d6c0 2023-07-08T12:39:17.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb678468700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700aca30 con 0x7fb6700ce1b0 2023-07-08T12:39:17.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb678468700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700ad760 con 0x7fb6700b1c70 2023-07-08T12:39:17.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb676a05700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb67015d6c0 0x7fb6701f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb676a05700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb67015d6c0 0x7fb6701f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51306/0 (socket says 172.21.15.12:51306) 2023-07-08T12:39:17.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb676a05700 1 -- 172.21.15.12:0/3348648440 learned_addr learned my addr 172.21.15.12:0/3348648440 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:17.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb675a03700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6700ce1b0 0x7fb6700adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb676204700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6700b1c70 0x7fb67014ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb676a05700 1 -- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6700b1c70 msgr2=0x7fb67014ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb676a05700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6700b1c70 0x7fb67014ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb676a05700 1 -- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6700ce1b0 msgr2=0x7fb6700adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb676a05700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6700ce1b0 0x7fb6700adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb676a05700 1 -- 172.21.15.12:0/3348648440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb668009a70 con 0x7fb67015d6c0 2023-07-08T12:39:17.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb675a03700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6700ce1b0 0x7fb6700adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:17.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb676a05700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb67015d6c0 0x7fb6701f5050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb668009e60 tx=0x7fb66800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:17.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb666ffd700 1 -- 172.21.15.12:0/3348648440 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668009cd0 con 0x7fb67015d6c0 2023-07-08T12:39:17.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb666ffd700 1 -- 172.21.15.12:0/3348648440 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66800f040 con 0x7fb67015d6c0 2023-07-08T12:39:17.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb666ffd700 1 -- 172.21.15.12:0/3348648440 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66800ce20 con 0x7fb67015d6c0 2023-07-08T12:39:17.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.486+0000 7fb678468700 1 -- 172.21.15.12:0/3348648440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6600094a0 con 0x7fb67015d6c0 2023-07-08T12:39:17.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.487+0000 7fb678468700 1 -- 172.21.15.12:0/3348648440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6701f5ef0 con 0x7fb67015d6c0 2023-07-08T12:39:17.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.488+0000 7fb678468700 1 -- 172.21.15.12:0/3348648440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67004ea60 con 0x7fb67015d6c0 2023-07-08T12:39:17.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.488+0000 7fb666ffd700 1 -- 172.21.15.12:0/3348648440 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb668014480 con 0x7fb67015d6c0 2023-07-08T12:39:17.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.488+0000 7fb666ffd700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb65406ace0 0x7fb65406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.488+0000 7fb666ffd700 1 -- 172.21.15.12:0/3348648440 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb668089ed0 con 0x7fb67015d6c0 2023-07-08T12:39:17.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.489+0000 7fb676204700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb65406ace0 0x7fb65406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.489+0000 7fb676204700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb65406ace0 0x7fb65406d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb66c001eb0 tx=0x7fb66c001ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:17.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.493+0000 7fb666ffd700 1 -- 172.21.15.12:0/3348648440 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb668054c60 con 0x7fb67015d6c0 2023-07-08T12:39:17.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.735+0000 7fb678468700 1 -- 172.21.15.12:0/3348648440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6700ad8a0 con 0x7fb67015d6c0 2023-07-08T12:39:17.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.735+0000 7fb666ffd700 1 -- 172.21.15.12:0/3348648440 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fb668058280 con 0x7fb67015d6c0 2023-07-08T12:39:17.738 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:17.738 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3238006310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3238006310},{"type":"v1","addr":"172.21.15.160:6836","nonce":3238006310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3864661454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3864661454},{"type":"v1","addr":"172.21.15.160:6837","nonce":3864661454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6834/946118830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":946118830},{"type":"v1","addr":"172.21.15.12:6834","nonce":946118830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-08T12:38:07.943202+0000","modified":"2023-07-08T12:38:32.319568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6835/3301134992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3301134992},{"type":"v1","addr":"172.21.15.12:6835","nonce":3301134992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-08T12:39:17.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.737+0000 7fb678468700 1 -- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb65406ace0 msgr2=0x7fb65406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.737+0000 7fb678468700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb65406ace0 0x7fb65406d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb66c001eb0 tx=0x7fb66c001ce0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.737+0000 7fb678468700 1 -- 172.21.15.12:0/3348648440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb67015d6c0 msgr2=0x7fb6701f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.737+0000 7fb678468700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb67015d6c0 0x7fb6701f5050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb668009e60 tx=0x7fb66800c3a0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.737+0000 7fb678468700 1 -- 172.21.15.12:0/3348648440 shutdown_connections 2023-07-08T12:39:17.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.737+0000 7fb678468700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb65406ace0 0x7fb65406d190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.737+0000 7fb678468700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb67015d6c0 0x7fb6701f5050 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.737+0000 7fb678468700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6700b1c70 0x7fb67014ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.737+0000 7fb678468700 1 --2- 172.21.15.12:0/3348648440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6700ce1b0 0x7fb6700adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.737+0000 7fb678468700 1 -- 172.21.15.12:0/3348648440 >> 172.21.15.12:0/3348648440 conn(0x7fb6700cc290 msgr2=0x7fb6700af270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.738+0000 7fb678468700 1 -- 172.21.15.12:0/3348648440 shutdown_connections 2023-07-08T12:39:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.738+0000 7fb678468700 1 -- 172.21.15.12:0/3348648440 wait complete. 2023-07-08T12:39:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2023-07-08T12:39:17.752 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:39:17.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.874+0000 7f8da0058700 1 Processor -- start 2023-07-08T12:39:17.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.874+0000 7f8da0058700 1 -- start start 2023-07-08T12:39:17.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.875+0000 7f8da0058700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8d980b8320 0x7f8d980cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.875+0000 7f8da0058700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8d980b4cc0 0x7f8d981608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.875+0000 7f8da0058700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d980cf3b0 con 0x7f8d980d1cd0 2023-07-08T12:39:17.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.875+0000 7f8da0058700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d98160e20 con 0x7f8d980b8320 2023-07-08T12:39:17.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.875+0000 7f8da0058700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d980b86f0 con 0x7f8d980b4cc0 2023-07-08T12:39:17.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.875+0000 7f8d9ddf4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8d980b8320 0x7f8d980cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.875+0000 7f8d9d5f3700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8d980d1cd0 0x7f8d980b45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38706/0 (socket says 172.21.15.12:38706) 2023-07-08T12:39:17.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.875+0000 7f8d9ddf4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8d980b8320 0x7f8d980cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51310/0 (socket says 172.21.15.12:51310) 2023-07-08T12:39:17.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.875+0000 7f8d9d5f3700 1 -- 172.21.15.12:0/3805210795 learned_addr learned my addr 172.21.15.12:0/3805210795 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:17.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.876+0000 7f8d9e5f5700 1 --2- 172.21.15.12:0/3805210795 >> v2:172.21.15.160:3300/0 conn(0x7f8d980b4cc0 0x7f8d981608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.876+0000 7f8d9ddf4700 1 -- 172.21.15.12:0/3805210795 >> v2:172.21.15.160:3300/0 conn(0x7f8d980b4cc0 msgr2=0x7f8d981608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:17.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.876+0000 7f8d9ddf4700 1 --2- 172.21.15.12:0/3805210795 >> v2:172.21.15.160:3300/0 conn(0x7f8d980b4cc0 0x7f8d981608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.876+0000 7f8d9ddf4700 1 -- 172.21.15.12:0/3805210795 >> v1:172.21.15.12:6789/0 conn(0x7f8d980d1cd0 legacy=0x7f8d980b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.876+0000 7f8d9cdf2700 1 -- 172.21.15.12:0/3805210795 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892256610 0 0) 0x7f8d980cf3b0 con 0x7f8d980d1cd0 2023-07-08T12:39:17.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.876+0000 7f8d9ddf4700 1 -- 172.21.15.12:0/3805210795 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d980cf0d0 con 0x7f8d980b8320 2023-07-08T12:39:17.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.876+0000 7f8d9ddf4700 1 --2- 172.21.15.12:0/3805210795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d980b8320 0x7f8d980cd4f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8d9806c630 tx=0x7f8d94009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9adc1aa366e6ce server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:17.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.876+0000 7f8d9cdf2700 1 -- 172.21.15.12:0/3805210795 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94011830 con 0x7f8d980b8320 2023-07-08T12:39:17.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.876+0000 7f8d9cdf2700 1 -- 172.21.15.12:0/3805210795 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d94011990 con 0x7f8d980b8320 2023-07-08T12:39:17.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.876+0000 7f8d9cdf2700 1 -- 172.21.15.12:0/3805210795 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94011d20 con 0x7f8d980b8320 2023-07-08T12:39:17.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.877+0000 7f8da0058700 1 -- 172.21.15.12:0/3805210795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d980b8320 msgr2=0x7f8d980cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.877+0000 7f8da0058700 1 --2- 172.21.15.12:0/3805210795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d980b8320 0x7f8d980cd4f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8d9806c630 tx=0x7f8d94009b20 comp rx=0 tx=0).stop 2023-07-08T12:39:17.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.877+0000 7f8da0058700 1 -- 172.21.15.12:0/3805210795 shutdown_connections 2023-07-08T12:39:17.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.877+0000 7f8da0058700 1 --2- 172.21.15.12:0/3805210795 >> v2:172.21.15.160:3300/0 conn(0x7f8d980b4cc0 0x7f8d981608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.877+0000 7f8da0058700 1 --2- 172.21.15.12:0/3805210795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d980b8320 0x7f8d980cd4f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.877+0000 7f8da0058700 1 -- 172.21.15.12:0/3805210795 >> 172.21.15.12:0/3805210795 conn(0x7f8d980722b0 msgr2=0x7f8d980cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:17.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.877+0000 7f8da0058700 1 -- 172.21.15.12:0/3805210795 shutdown_connections 2023-07-08T12:39:17.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.877+0000 7f8da0058700 1 -- 172.21.15.12:0/3805210795 wait complete. 2023-07-08T12:39:17.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.878+0000 7f8da0058700 1 Processor -- start 2023-07-08T12:39:17.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.878+0000 7f8da0058700 1 -- start start 2023-07-08T12:39:17.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8da0058700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d980b8320 0x7f8d980c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8da0058700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d980d1cd0 0x7f8d98152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8da0058700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8d981510b0 0x7f8d98159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8da0058700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d98154300 con 0x7f8d980b8320 2023-07-08T12:39:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8da0058700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d981536f0 con 0x7f8d980d1cd0 2023-07-08T12:39:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8da0058700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d981539d0 con 0x7f8d981510b0 2023-07-08T12:39:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8d9ddf4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d980b8320 0x7f8d980c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8d9ddf4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d980b8320 0x7f8d980c9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51316/0 (socket says 172.21.15.12:51316) 2023-07-08T12:39:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8d9ddf4700 1 -- 172.21.15.12:0/3096486094 learned_addr learned my addr 172.21.15.12:0/3096486094 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:17.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8d9d5f3700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d980d1cd0 0x7f8d98152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8d9e5f5700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8d981510b0 0x7f8d98159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8d9ddf4700 1 -- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8d981510b0 msgr2=0x7f8d98159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8d9ddf4700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8d981510b0 0x7f8d98159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8d9ddf4700 1 -- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d980d1cd0 msgr2=0x7f8d98152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:17.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8d9ddf4700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d980d1cd0 0x7f8d98152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:17.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8d9ddf4700 1 -- 172.21.15.12:0/3096486094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d940094a0 con 0x7f8d980b8320 2023-07-08T12:39:17.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.879+0000 7f8d9ddf4700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d980b8320 0x7f8d980c9010 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8d94000c00 tx=0x7f8d94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:17.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.880+0000 7f8d8a7fc700 1 -- 172.21.15.12:0/3096486094 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d940036a0 con 0x7f8d980b8320 2023-07-08T12:39:17.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.880+0000 7f8d8a7fc700 1 -- 172.21.15.12:0/3096486094 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9401e070 con 0x7f8d980b8320 2023-07-08T12:39:17.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.880+0000 7f8da0058700 1 -- 172.21.15.12:0/3096486094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d98153cb0 con 0x7f8d980b8320 2023-07-08T12:39:17.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.880+0000 7f8d8a7fc700 1 -- 172.21.15.12:0/3096486094 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94003a20 con 0x7f8d980b8320 2023-07-08T12:39:17.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.880+0000 7f8da0058700 1 -- 172.21.15.12:0/3096486094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d98154790 con 0x7f8d980b8320 2023-07-08T12:39:17.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.881+0000 7f8da0058700 1 -- 172.21.15.12:0/3096486094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d981545e0 con 0x7f8d980b8320 2023-07-08T12:39:17.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.881+0000 7f8d8a7fc700 1 -- 172.21.15.12:0/3096486094 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d94003b80 con 0x7f8d980b8320 2023-07-08T12:39:17.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.881+0000 7f8d8a7fc700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f8d7c06ace0 0x7f8d7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:17.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.881+0000 7f8d8a7fc700 1 -- 172.21.15.12:0/3096486094 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8d9408a840 con 0x7f8d980b8320 2023-07-08T12:39:17.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.882+0000 7f8d9d5f3700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f8d7c06ace0 0x7f8d7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:17.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.882+0000 7f8d9d5f3700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f8d7c06ace0 0x7f8d7c06d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d980c8470 tx=0x7f8d8c007be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:17.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:17.886+0000 7f8d8a7fc700 1 -- 172.21.15.12:0/3096486094 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8d94055650 con 0x7f8d980b8320 2023-07-08T12:39:18.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.085+0000 7f8da0058700 1 -- 172.21.15.12:0/3096486094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d9804ea60 con 0x7f8d980b8320 2023-07-08T12:39:18.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.086+0000 7f8d8a7fc700 1 -- 172.21.15.12:0/3096486094 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20203 (secure 0 0 0) 0x7f8d94020090 con 0x7f8d980b8320 2023-07-08T12:39:18.088 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:18.089 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:49.386462+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:39:18.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.088+0000 7f8da0058700 1 -- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f8d7c06ace0 msgr2=0x7f8d7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:18.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.088+0000 7f8da0058700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f8d7c06ace0 0x7f8d7c06d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d980c8470 tx=0x7f8d8c007be0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.088+0000 7f8da0058700 1 -- 172.21.15.12:0/3096486094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d980b8320 msgr2=0x7f8d980c9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:18.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.088+0000 7f8da0058700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d980b8320 0x7f8d980c9010 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8d94000c00 tx=0x7f8d94004750 comp rx=0 tx=0).stop 2023-07-08T12:39:18.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.088+0000 7f8da0058700 1 -- 172.21.15.12:0/3096486094 shutdown_connections 2023-07-08T12:39:18.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.088+0000 7f8da0058700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f8d7c06ace0 0x7f8d7c06d190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.088+0000 7f8da0058700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8d981510b0 0x7f8d98159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.088+0000 7f8da0058700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d980b8320 0x7f8d980c9010 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.088+0000 7f8da0058700 1 --2- 172.21.15.12:0/3096486094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d980d1cd0 0x7f8d98152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.088+0000 7f8da0058700 1 -- 172.21.15.12:0/3096486094 >> 172.21.15.12:0/3096486094 conn(0x7f8d980b4cc0 msgr2=0x7f8d9815c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:18.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.088+0000 7f8da0058700 1 -- 172.21.15.12:0/3096486094 shutdown_connections 2023-07-08T12:39:18.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.088+0000 7f8da0058700 1 -- 172.21.15.12:0/3096486094 wait complete. 2023-07-08T12:39:18.103 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-08T12:39:18.103 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-08T12:39:18.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.225+0000 7f41ead5e700 1 Processor -- start 2023-07-08T12:39:18.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.226+0000 7f41ead5e700 1 -- start start 2023-07-08T12:39:18.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.226+0000 7f41ead5e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41e4159c30 0x7f41e40cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:18.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.226+0000 7f41ead5e700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f41e4160790 0x7f41e406ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:18.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.226+0000 7f41ead5e700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e415c890 con 0x7f41e4069f30 2023-07-08T12:39:18.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.226+0000 7f41ead5e700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e415c730 con 0x7f41e4160790 2023-07-08T12:39:18.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.226+0000 7f41ead5e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e40cf590 con 0x7f41e4159c30 2023-07-08T12:39:18.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.227+0000 7f41e8afa700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41e4159c30 0x7f41e40cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.227+0000 7f41e8afa700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41e4159c30 0x7f41e40cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:60942/0 (socket says 172.21.15.12:60942) 2023-07-08T12:39:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.227+0000 7f41e8afa700 1 -- 172.21.15.12:0/3155120264 learned_addr learned my addr 172.21.15.12:0/3155120264 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.227+0000 7f41e3fff700 1 --2- 172.21.15.12:0/3155120264 >> v2:172.21.15.160:3300/0 conn(0x7f41e4160790 0x7f41e406ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:18.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.227+0000 7f41e92fb700 1 --1- 172.21.15.12:0/3155120264 >> v1:172.21.15.160:6789/0 conn(0x7f41e4069f30 0x7f41e415cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:47946/0 (socket says 172.21.15.12:47946) 2023-07-08T12:39:18.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.227+0000 7f41e37fe700 1 -- 172.21.15.12:0/3155120264 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 639208143 0 0) 0x7f41e415c890 con 0x7f41e4069f30 2023-07-08T12:39:18.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.227+0000 7f41e8afa700 1 -- 172.21.15.12:0/3155120264 >> v2:172.21.15.160:3300/0 conn(0x7f41e4160790 msgr2=0x7f41e406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:18.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.227+0000 7f41e8afa700 1 --2- 172.21.15.12:0/3155120264 >> v2:172.21.15.160:3300/0 conn(0x7f41e4160790 0x7f41e406ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.227+0000 7f41e8afa700 1 -- 172.21.15.12:0/3155120264 >> v1:172.21.15.160:6789/0 conn(0x7f41e4069f30 legacy=0x7f41e415cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.227+0000 7f41e8afa700 1 -- 172.21.15.12:0/3155120264 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e4160c90 con 0x7f41e4159c30 2023-07-08T12:39:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.227+0000 7f41e8afa700 1 --2- 172.21.15.12:0/3155120264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41e4159c30 0x7f41e40cf050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f41d80098f0 tx=0x7f41d8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb8d61c70feaecb0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:18.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.228+0000 7f41e37fe700 1 -- 172.21.15.12:0/3155120264 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d8011830 con 0x7f41e4159c30 2023-07-08T12:39:18.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.228+0000 7f41e37fe700 1 -- 172.21.15.12:0/3155120264 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d8011990 con 0x7f41e4159c30 2023-07-08T12:39:18.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.228+0000 7f41e37fe700 1 -- 172.21.15.12:0/3155120264 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d8011d20 con 0x7f41e4159c30 2023-07-08T12:39:18.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.228+0000 7f41ead5e700 1 -- 172.21.15.12:0/3155120264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41e4159c30 msgr2=0x7f41e40cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.228+0000 7f41ead5e700 1 --2- 172.21.15.12:0/3155120264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41e4159c30 0x7f41e40cf050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f41d80098f0 tx=0x7f41d8009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.228+0000 7f41ead5e700 1 -- 172.21.15.12:0/3155120264 shutdown_connections 2023-07-08T12:39:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.228+0000 7f41ead5e700 1 --2- 172.21.15.12:0/3155120264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41e4159c30 0x7f41e40cf050 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.228+0000 7f41ead5e700 1 --2- 172.21.15.12:0/3155120264 >> v2:172.21.15.160:3300/0 conn(0x7f41e4160790 0x7f41e406ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.228+0000 7f41ead5e700 1 -- 172.21.15.12:0/3155120264 >> 172.21.15.12:0/3155120264 conn(0x7f41e40cc290 msgr2=0x7f41e40cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.228+0000 7f41ead5e700 1 -- 172.21.15.12:0/3155120264 shutdown_connections 2023-07-08T12:39:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.228+0000 7f41ead5e700 1 -- 172.21.15.12:0/3155120264 wait complete. 2023-07-08T12:39:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.229+0000 7f41ead5e700 1 Processor -- start 2023-07-08T12:39:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.229+0000 7f41ead5e700 1 -- start start 2023-07-08T12:39:18.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.230+0000 7f41ead5e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41e4069f30 0x7f41e4151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:18.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.230+0000 7f41ead5e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41e4159c30 0x7f41e40b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:18.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.230+0000 7f41ead5e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41e4160790 0x7f41e40cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:18.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.230+0000 7f41ead5e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e41e8300 con 0x7f41e4159c30 2023-07-08T12:39:18.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.230+0000 7f41ead5e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e40b4770 con 0x7f41e4160790 2023-07-08T12:39:18.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.230+0000 7f41ead5e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e41e7960 con 0x7f41e4069f30 2023-07-08T12:39:18.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.230+0000 7f41e3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41e4159c30 0x7f41e40b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:18.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41e4159c30 0x7f41e40b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51332/0 (socket says 172.21.15.12:51332) 2023-07-08T12:39:18.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e3fff700 1 -- 172.21.15.12:0/2812598992 learned_addr learned my addr 172.21.15.12:0/2812598992 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:18.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e8afa700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41e4069f30 0x7f41e4151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:18.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e92fb700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41e4160790 0x7f41e40cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:18.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e3fff700 1 -- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41e4069f30 msgr2=0x7f41e4151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:18.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e3fff700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41e4069f30 0x7f41e4151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e3fff700 1 -- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41e4160790 msgr2=0x7f41e40cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:18.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e3fff700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41e4160790 0x7f41e40cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e3fff700 1 -- 172.21.15.12:0/2812598992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d8009550 con 0x7f41e4159c30 2023-07-08T12:39:18.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e92fb700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41e4160790 0x7f41e40cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:18.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e3fff700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41e4159c30 0x7f41e40b3d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f41d000e840 tx=0x7f41d000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:18.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e17fa700 1 -- 172.21.15.12:0/2812598992 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0009590 con 0x7f41e4159c30 2023-07-08T12:39:18.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e17fa700 1 -- 172.21.15.12:0/2812598992 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d0013680 con 0x7f41e4159c30 2023-07-08T12:39:18.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41ead5e700 1 -- 172.21.15.12:0/2812598992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e41e7c40 con 0x7f41e4159c30 2023-07-08T12:39:18.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41e17fa700 1 -- 172.21.15.12:0/2812598992 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0013a30 con 0x7f41e4159c30 2023-07-08T12:39:18.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.231+0000 7f41ead5e700 1 -- 172.21.15.12:0/2812598992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e41e8720 con 0x7f41e4159c30 2023-07-08T12:39:18.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.232+0000 7f41ead5e700 1 -- 172.21.15.12:0/2812598992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e404ea60 con 0x7f41e4159c30 2023-07-08T12:39:18.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.233+0000 7f41e17fa700 1 -- 172.21.15.12:0/2812598992 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41d0013b90 con 0x7f41e4159c30 2023-07-08T12:39:18.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.233+0000 7f41e17fa700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f41c806ace0 0x7f41c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:18.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.233+0000 7f41e17fa700 1 -- 172.21.15.12:0/2812598992 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f41d00895f0 con 0x7f41e4159c30 2023-07-08T12:39:18.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.233+0000 7f41e8afa700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f41c806ace0 0x7f41c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:18.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.234+0000 7f41e8afa700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f41c806ace0 0x7f41c806d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f41d8000c00 tx=0x7f41d8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:18.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.237+0000 7f41e17fa700 1 -- 172.21.15.12:0/2812598992 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f41d0057e90 con 0x7f41e4159c30 2023-07-08T12:39:18.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.479+0000 7f41ead5e700 1 -- 172.21.15.12:0/2812598992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41e40b5790 con 0x7f41e4159c30 2023-07-08T12:39:18.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.480+0000 7f41e17fa700 1 -- 172.21.15.12:0/2812598992 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f41d0057a20 con 0x7f41e4159c30 2023-07-08T12:39:18.482 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:18.483 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3238006310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3238006310},{"type":"v1","addr":"172.21.15.160:6836","nonce":3238006310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3864661454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3864661454},{"type":"v1","addr":"172.21.15.160:6837","nonce":3864661454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6834/946118830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":946118830},{"type":"v1","addr":"172.21.15.12:6834","nonce":946118830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-08T12:38:07.943202+0000","modified":"2023-07-08T12:38:32.319568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6835/3301134992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3301134992},{"type":"v1","addr":"172.21.15.12:6835","nonce":3301134992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-08T12:39:18.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.482+0000 7f41ead5e700 1 -- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f41c806ace0 msgr2=0x7f41c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:18.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.482+0000 7f41ead5e700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f41c806ace0 0x7f41c806d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f41d8000c00 tx=0x7f41d8008030 comp rx=0 tx=0).stop 2023-07-08T12:39:18.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.482+0000 7f41ead5e700 1 -- 172.21.15.12:0/2812598992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41e4159c30 msgr2=0x7f41e40b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:18.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.482+0000 7f41ead5e700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41e4159c30 0x7f41e40b3d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f41d000e840 tx=0x7f41d000eb50 comp rx=0 tx=0).stop 2023-07-08T12:39:18.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.482+0000 7f41ead5e700 1 -- 172.21.15.12:0/2812598992 shutdown_connections 2023-07-08T12:39:18.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.482+0000 7f41ead5e700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f41c806ace0 0x7f41c806d190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.482+0000 7f41ead5e700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41e4160790 0x7f41e40cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.482+0000 7f41ead5e700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41e4069f30 0x7f41e4151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.482+0000 7f41ead5e700 1 --2- 172.21.15.12:0/2812598992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41e4159c30 0x7f41e40b3d90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.482+0000 7f41ead5e700 1 -- 172.21.15.12:0/2812598992 >> 172.21.15.12:0/2812598992 conn(0x7f41e40cc290 msgr2=0x7f41e415ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:18.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.482+0000 7f41ead5e700 1 -- 172.21.15.12:0/2812598992 shutdown_connections 2023-07-08T12:39:18.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.482+0000 7f41ead5e700 1 -- 172.21.15.12:0/2812598992 wait complete. 2023-07-08T12:39:18.486 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2023-07-08T12:39:18.496 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-08T12:39:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.619+0000 7f30f1945700 1 Processor -- start 2023-07-08T12:39:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.619+0000 7f30f1945700 1 -- start start 2023-07-08T12:39:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.620+0000 7f30f1945700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f30ec15a3a0 0x7f30ec15e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.620+0000 7f30f1945700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f30ec0ca2e0 0x7f30ec06ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.620+0000 7f30f1945700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ec15a770 con 0x7f30ec0c9e60 2023-07-08T12:39:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.620+0000 7f30f1945700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ec15e500 con 0x7f30ec15a3a0 2023-07-08T12:39:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.620+0000 7f30f1945700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ec0cbd10 con 0x7f30ec0ca2e0 2023-07-08T12:39:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.620+0000 7f30eaffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f30ec15a3a0 0x7f30ec15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.620+0000 7f30eaffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f30ec15a3a0 0x7f30ec15e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51338/0 (socket says 172.21.15.12:51338) 2023-07-08T12:39:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.620+0000 7f30eaffd700 1 -- 172.21.15.12:0/2463441541 learned_addr learned my addr 172.21.15.12:0/2463441541 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.621+0000 7f30eb7fe700 1 --2- 172.21.15.12:0/2463441541 >> v2:172.21.15.160:3300/0 conn(0x7f30ec0ca2e0 0x7f30ec06ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.621+0000 7f30eaffd700 1 -- 172.21.15.12:0/2463441541 >> v2:172.21.15.160:3300/0 conn(0x7f30ec0ca2e0 msgr2=0x7f30ec06ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.621+0000 7f30eaffd700 1 --2- 172.21.15.12:0/2463441541 >> v2:172.21.15.160:3300/0 conn(0x7f30ec0ca2e0 0x7f30ec06ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.621+0000 7f30eaffd700 1 -- 172.21.15.12:0/2463441541 >> v1:172.21.15.160:6789/0 conn(0x7f30ec0c9e60 legacy=0x7f30ec069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.621+0000 7f30eaffd700 1 -- 172.21.15.12:0/2463441541 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec15deb0 con 0x7f30ec15a3a0 2023-07-08T12:39:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.621+0000 7f30eaffd700 1 --2- 172.21.15.12:0/2463441541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec15a3a0 0x7f30ec15e990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0aec90 tx=0x7f30e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6785d0ab66b0313 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.621+0000 7f30e9ffb700 1 -- 172.21.15.12:0/2463441541 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e0004230 con 0x7f30ec15a3a0 2023-07-08T12:39:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.621+0000 7f30e9ffb700 1 -- 172.21.15.12:0/2463441541 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e0004390 con 0x7f30ec15a3a0 2023-07-08T12:39:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.621+0000 7f30e9ffb700 1 -- 172.21.15.12:0/2463441541 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e00036a0 con 0x7f30ec15a3a0 2023-07-08T12:39:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.622+0000 7f30f1945700 1 -- 172.21.15.12:0/2463441541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec15a3a0 msgr2=0x7f30ec15e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.622+0000 7f30f1945700 1 --2- 172.21.15.12:0/2463441541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec15a3a0 0x7f30ec15e990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0aec90 tx=0x7f30e0009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.622+0000 7f30f1945700 1 -- 172.21.15.12:0/2463441541 shutdown_connections 2023-07-08T12:39:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.622+0000 7f30f1945700 1 --2- 172.21.15.12:0/2463441541 >> v2:172.21.15.160:3300/0 conn(0x7f30ec0ca2e0 0x7f30ec06ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.622+0000 7f30f1945700 1 --2- 172.21.15.12:0/2463441541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec15a3a0 0x7f30ec15e990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.622+0000 7f30f1945700 1 -- 172.21.15.12:0/2463441541 >> 172.21.15.12:0/2463441541 conn(0x7f30ec0cc290 msgr2=0x7f30ec0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.622+0000 7f30f1945700 1 -- 172.21.15.12:0/2463441541 shutdown_connections 2023-07-08T12:39:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.622+0000 7f30f1945700 1 -- 172.21.15.12:0/2463441541 wait complete. 2023-07-08T12:39:18.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.623+0000 7f30f1945700 1 Processor -- start 2023-07-08T12:39:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.623+0000 7f30f1945700 1 -- start start 2023-07-08T12:39:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30f1945700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec0c9e60 0x7f30ec145870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30f1945700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30ec0ca2e0 0x7f30ec1efd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30f1945700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30ec15a3a0 0x7f30ec0b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30f1945700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec0c6980 con 0x7f30ec0c9e60 2023-07-08T12:39:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30f1945700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec0c6c60 con 0x7f30ec0ca2e0 2023-07-08T12:39:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30f1945700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec0c6f40 con 0x7f30ec15a3a0 2023-07-08T12:39:18.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30eaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec0c9e60 0x7f30ec145870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:18.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30eaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec0c9e60 0x7f30ec145870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51344/0 (socket says 172.21.15.12:51344) 2023-07-08T12:39:18.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30eaffd700 1 -- 172.21.15.12:0/2113715782 learned_addr learned my addr 172.21.15.12:0/2113715782 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:18.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30eb7fe700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30ec15a3a0 0x7f30ec0b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:18.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30ea7fc700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30ec0ca2e0 0x7f30ec1efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:18.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30eaffd700 1 -- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30ec15a3a0 msgr2=0x7f30ec0b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:18.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30eaffd700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30ec15a3a0 0x7f30ec0b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30eaffd700 1 -- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30ec0ca2e0 msgr2=0x7f30ec1efd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:18.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30eaffd700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30ec0ca2e0 0x7f30ec1efd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30eaffd700 1 -- 172.21.15.12:0/2113715782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e0009550 con 0x7f30ec0c9e60 2023-07-08T12:39:18.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30ea7fc700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30ec0ca2e0 0x7f30ec1efd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:18.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30eb7fe700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30ec15a3a0 0x7f30ec0b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:18.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.624+0000 7f30eaffd700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec0c9e60 0x7f30ec145870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009130 tx=0x7f30e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:18.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.625+0000 7f30d3fff700 1 -- 172.21.15.12:0/2113715782 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e0011630 con 0x7f30ec0c9e60 2023-07-08T12:39:18.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.625+0000 7f30d3fff700 1 -- 172.21.15.12:0/2113715782 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e0011790 con 0x7f30ec0c9e60 2023-07-08T12:39:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.625+0000 7f30f1945700 1 -- 172.21.15.12:0/2113715782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ec0c7220 con 0x7f30ec0c9e60 2023-07-08T12:39:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.625+0000 7f30d3fff700 1 -- 172.21.15.12:0/2113715782 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e0011b40 con 0x7f30ec0c9e60 2023-07-08T12:39:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.625+0000 7f30f1945700 1 -- 172.21.15.12:0/2113715782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ec1f0240 con 0x7f30ec0c9e60 2023-07-08T12:39:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.626+0000 7f30f1945700 1 -- 172.21.15.12:0/2113715782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ec0c7740 con 0x7f30ec0c9e60 2023-07-08T12:39:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.626+0000 7f30d3fff700 1 -- 172.21.15.12:0/2113715782 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f30e001b440 con 0x7f30ec0c9e60 2023-07-08T12:39:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.627+0000 7f30d3fff700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f30d806ace0 0x7f30d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.627+0000 7f30d3fff700 1 -- 172.21.15.12:0/2113715782 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f30e008ace0 con 0x7f30ec0c9e60 2023-07-08T12:39:18.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.627+0000 7f30ea7fc700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f30d806ace0 0x7f30d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:18.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.627+0000 7f30ea7fc700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f30d806ace0 0x7f30d806d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f30ec15c1b0 tx=0x7f30dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:18.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.631+0000 7f30d3fff700 1 -- 172.21.15.12:0/2113715782 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f30e0055af0 con 0x7f30ec0c9e60 2023-07-08T12:39:18.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.877+0000 7f30f1945700 1 -- 172.21.15.12:0/2113715782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30ec04ea60 con 0x7f30ec0c9e60 2023-07-08T12:39:18.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.877+0000 7f30d3fff700 1 -- 172.21.15.12:0/2113715782 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f30e0059110 con 0x7f30ec0c9e60 2023-07-08T12:39:18.880 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:18.881 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3238006310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3238006310},{"type":"v1","addr":"172.21.15.160:6836","nonce":3238006310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3864661454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3864661454},{"type":"v1","addr":"172.21.15.160:6837","nonce":3864661454}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6834/946118830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":946118830},{"type":"v1","addr":"172.21.15.12:6834","nonce":946118830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-08T12:38:07.943202+0000","modified":"2023-07-08T12:38:32.319568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6835/3301134992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":3301134992},{"type":"v1","addr":"172.21.15.12:6835","nonce":3301134992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-08T12:39:18.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.879+0000 7f30f1945700 1 -- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f30d806ace0 msgr2=0x7f30d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:18.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.879+0000 7f30f1945700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f30d806ace0 0x7f30d806d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f30ec15c1b0 tx=0x7f30dc009040 comp rx=0 tx=0).stop 2023-07-08T12:39:18.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.879+0000 7f30f1945700 1 -- 172.21.15.12:0/2113715782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec0c9e60 msgr2=0x7f30ec145870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:18.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.879+0000 7f30f1945700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec0c9e60 0x7f30ec145870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009130 tx=0x7f30e0004750 comp rx=0 tx=0).stop 2023-07-08T12:39:18.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.879+0000 7f30f1945700 1 -- 172.21.15.12:0/2113715782 shutdown_connections 2023-07-08T12:39:18.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.879+0000 7f30f1945700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f30d806ace0 0x7f30d806d190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.879+0000 7f30f1945700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30ec15a3a0 0x7f30ec0b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.879+0000 7f30f1945700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30ec0c9e60 0x7f30ec145870 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.879+0000 7f30f1945700 1 --2- 172.21.15.12:0/2113715782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30ec0ca2e0 0x7f30ec1efd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:18.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.879+0000 7f30f1945700 1 -- 172.21.15.12:0/2113715782 >> 172.21.15.12:0/2113715782 conn(0x7f30ec0cc290 msgr2=0x7f30ec0afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:18.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.879+0000 7f30f1945700 1 -- 172.21.15.12:0/2113715782 shutdown_connections 2023-07-08T12:39:18.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:18.879+0000 7f30f1945700 1 -- 172.21.15.12:0/2113715782 wait complete. 2023-07-08T12:39:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2023-07-08T12:39:18.894 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:39:19.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.017+0000 7fa280cae700 1 Processor -- start 2023-07-08T12:39:19.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.017+0000 7fa280cae700 1 -- start start 2023-07-08T12:39:19.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.018+0000 7fa280cae700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa27c0cd7a0 0x7fa27c0cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.018+0000 7fa280cae700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa27c0caf90 0x7fa27c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.018+0000 7fa280cae700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27c1594b0 con 0x7fa27c0ccff0 2023-07-08T12:39:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.018+0000 7fa280cae700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27c0ce0b0 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.018+0000 7fa280cae700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27c0ca1d0 con 0x7fa27c0caf90 2023-07-08T12:39:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.018+0000 7fa27a59c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa27c0cd7a0 0x7fa27c0cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.018+0000 7fa27ad9d700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa27c0ccff0 0x7fa27c069f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38752/0 (socket says 172.21.15.12:38752) 2023-07-08T12:39:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.018+0000 7fa27ad9d700 1 -- 172.21.15.12:0/1947435212 learned_addr learned my addr 172.21.15.12:0/1947435212 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.018+0000 7fa279d9b700 1 --2- 172.21.15.12:0/1947435212 >> v2:172.21.15.160:3300/0 conn(0x7fa27c0caf90 0x7fa27c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.019+0000 7fa27a59c700 1 -- 172.21.15.12:0/1947435212 >> v2:172.21.15.160:3300/0 conn(0x7fa27c0caf90 msgr2=0x7fa27c0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.019+0000 7fa27a59c700 1 --2- 172.21.15.12:0/1947435212 >> v2:172.21.15.160:3300/0 conn(0x7fa27c0caf90 0x7fa27c0ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.019+0000 7fa27a59c700 1 -- 172.21.15.12:0/1947435212 >> v1:172.21.15.12:6789/0 conn(0x7fa27c0ccff0 legacy=0x7fa27c069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.019+0000 7fa27a59c700 1 -- 172.21.15.12:0/1947435212 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c0ca4b0 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.019+0000 7fa27959a700 1 -- 172.21.15.12:0/1947435212 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068714076 0 0) 0x7fa27c1594b0 con 0x7fa27c0ccff0 2023-07-08T12:39:19.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.019+0000 7fa27a59c700 1 --2- 172.21.15.12:0/1947435212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa27c0cd7a0 0x7fa27c0cdb70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa27c06bb20 tx=0x7fa270009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f55040f4fde3cc server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:19.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.019+0000 7fa27959a700 1 -- 172.21.15.12:0/1947435212 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270011830 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.019+0000 7fa27959a700 1 -- 172.21.15.12:0/1947435212 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa270011990 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.019+0000 7fa27959a700 1 -- 172.21.15.12:0/1947435212 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270011d20 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.019+0000 7fa280cae700 1 -- 172.21.15.12:0/1947435212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa27c0cd7a0 msgr2=0x7fa27c0cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.019+0000 7fa280cae700 1 --2- 172.21.15.12:0/1947435212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa27c0cd7a0 0x7fa27c0cdb70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa27c06bb20 tx=0x7fa270009b20 comp rx=0 tx=0).stop 2023-07-08T12:39:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.019+0000 7fa280cae700 1 -- 172.21.15.12:0/1947435212 shutdown_connections 2023-07-08T12:39:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.020+0000 7fa280cae700 1 --2- 172.21.15.12:0/1947435212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa27c0cd7a0 0x7fa27c0cdb70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.020+0000 7fa280cae700 1 --2- 172.21.15.12:0/1947435212 >> v2:172.21.15.160:3300/0 conn(0x7fa27c0caf90 0x7fa27c0ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.020+0000 7fa280cae700 1 -- 172.21.15.12:0/1947435212 >> 172.21.15.12:0/1947435212 conn(0x7fa27c0cb9f0 msgr2=0x7fa27c0cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.020+0000 7fa280cae700 1 -- 172.21.15.12:0/1947435212 shutdown_connections 2023-07-08T12:39:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.020+0000 7fa280cae700 1 -- 172.21.15.12:0/1947435212 wait complete. 2023-07-08T12:39:19.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.021+0000 7fa280cae700 1 Processor -- start 2023-07-08T12:39:19.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.021+0000 7fa280cae700 1 -- start start 2023-07-08T12:39:19.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa280cae700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa27c0caf90 0x7fa27c1542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa280cae700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa27c0ccff0 0x7fa27c15d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa280cae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa27c0cd7a0 0x7fa27c0c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa280cae700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0aefe0 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa280cae700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0c4e90 con 0x7fa27c0ccff0 2023-07-08T12:39:19.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa280cae700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0c5170 con 0x7fa27c0caf90 2023-07-08T12:39:19.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa27ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa27c0cd7a0 0x7fa27c0c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa27ad9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa27c0cd7a0 0x7fa27c0c7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51362/0 (socket says 172.21.15.12:51362) 2023-07-08T12:39:19.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa27ad9d700 1 -- 172.21.15.12:0/1085669585 learned_addr learned my addr 172.21.15.12:0/1085669585 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:19.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa27ad9d700 1 -- 172.21.15.12:0/1085669585 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa27c0caf90 msgr2=0x7fa27c1542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa279d9b700 1 --2- 172.21.15.12:0/1085669585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa27c0ccff0 0x7fa27c15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa27ad9d700 1 --2- 172.21.15.12:0/1085669585 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa27c0caf90 0x7fa27c1542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa27ad9d700 1 -- 172.21.15.12:0/1085669585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa27c0ccff0 msgr2=0x7fa27c15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa27ad9d700 1 --2- 172.21.15.12:0/1085669585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa27c0ccff0 0x7fa27c15d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa27ad9d700 1 -- 172.21.15.12:0/1085669585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2700094a0 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa27ad9d700 1 --2- 172.21.15.12:0/1085669585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa27c0cd7a0 0x7fa27c0c7000 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa274002230 tx=0x7fa27400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa26affd700 1 -- 172.21.15.12:0/1085669585 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27400ac20 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa26affd700 1 -- 172.21.15.12:0/1085669585 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27400ada0 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa280cae700 1 -- 172.21.15.12:0/1085669585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c0ae990 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.022+0000 7fa26affd700 1 -- 172.21.15.12:0/1085669585 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274007310 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.023+0000 7fa280cae700 1 -- 172.21.15.12:0/1085669585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c1547e0 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.023+0000 7fa280cae700 1 -- 172.21.15.12:0/1085669585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c04ea60 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.024+0000 7fa26affd700 1 -- 172.21.15.12:0/1085669585 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa274007470 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.024+0000 7fa26affd700 1 --2- 172.21.15.12:0/1085669585 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa25c06ace0 0x7fa25c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.024+0000 7fa26affd700 1 -- 172.21.15.12:0/1085669585 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa274089760 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.025+0000 7fa27a59c700 1 --2- 172.21.15.12:0/1085669585 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa25c06ace0 0x7fa25c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.025+0000 7fa27a59c700 1 --2- 172.21.15.12:0/1085669585 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa25c06ace0 0x7fa25c06d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa27c1f10a0 tx=0x7fa270010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:19.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.029+0000 7fa26affd700 1 -- 172.21.15.12:0/1085669585 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa274054450 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.227+0000 7fa280cae700 1 -- 172.21.15.12:0/1085669585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa27c06db30 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.229+0000 7fa26affd700 1 -- 172.21.15.12:0/1085669585 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+20203 (secure 0 0 0) 0x7fa274018090 con 0x7fa27c0cd7a0 2023-07-08T12:39:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:19.232 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:38:49.386462+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:38:05.660194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-08T12:38:06.920177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-07-08T12:38:10.592749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:39:19.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.230+0000 7fa280cae700 1 -- 172.21.15.12:0/1085669585 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa25c06ace0 msgr2=0x7fa25c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.231+0000 7fa280cae700 1 --2- 172.21.15.12:0/1085669585 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa25c06ace0 0x7fa25c06d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa27c1f10a0 tx=0x7fa270010000 comp rx=0 tx=0).stop 2023-07-08T12:39:19.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.231+0000 7fa280cae700 1 -- 172.21.15.12:0/1085669585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa27c0cd7a0 msgr2=0x7fa27c0c7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.231+0000 7fa280cae700 1 --2- 172.21.15.12:0/1085669585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa27c0cd7a0 0x7fa27c0c7000 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa274002230 tx=0x7fa27400a060 comp rx=0 tx=0).stop 2023-07-08T12:39:19.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.231+0000 7fa280cae700 1 -- 172.21.15.12:0/1085669585 shutdown_connections 2023-07-08T12:39:19.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.231+0000 7fa280cae700 1 --2- 172.21.15.12:0/1085669585 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fa25c06ace0 0x7fa25c06d190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.231+0000 7fa280cae700 1 --2- 172.21.15.12:0/1085669585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa27c0cd7a0 0x7fa27c0c7000 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.231+0000 7fa280cae700 1 --2- 172.21.15.12:0/1085669585 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa27c0caf90 0x7fa27c1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.231+0000 7fa280cae700 1 --2- 172.21.15.12:0/1085669585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa27c0ccff0 0x7fa27c15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.231+0000 7fa280cae700 1 -- 172.21.15.12:0/1085669585 >> 172.21.15.12:0/1085669585 conn(0x7fa27c0cb9f0 msgr2=0x7fa27c0cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:19.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.231+0000 7fa280cae700 1 -- 172.21.15.12:0/1085669585 shutdown_connections 2023-07-08T12:39:19.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.231+0000 7fa280cae700 1 -- 172.21.15.12:0/1085669585 wait complete. 2023-07-08T12:39:19.245 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-08T12:39:19.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.369+0000 7f0e8db48700 1 Processor -- start 2023-07-08T12:39:19.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.369+0000 7f0e8db48700 1 -- start start 2023-07-08T12:39:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.370+0000 7f0e8db48700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0e8815e990 0x7f0e88069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.370+0000 7f0e8db48700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8815a770 con 0x7f0e8815a3a0 2023-07-08T12:39:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.370+0000 7f0e8db48700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8815e500 con 0x7f0e8806a470 2023-07-08T12:39:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.370+0000 7f0e8db48700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e880ca570 con 0x7f0e8815e990 2023-07-08T12:39:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.370+0000 7f0e86ffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0e8815e990 0x7f0e88069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.370+0000 7f0e877fe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0e8815a3a0 0x7f0e880c9e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38762/0 (socket says 172.21.15.12:38762) 2023-07-08T12:39:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.370+0000 7f0e877fe700 1 -- 172.21.15.12:0/1727348037 learned_addr learned my addr 172.21.15.12:0/1727348037 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.371+0000 7f0e86ffd700 1 -- 172.21.15.12:0/1727348037 >> v1:172.21.15.160:6789/0 conn(0x7f0e8806a470 legacy=0x7f0e8806ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.371+0000 7f0e86ffd700 1 -- 172.21.15.12:0/1727348037 >> v1:172.21.15.12:6789/0 conn(0x7f0e8815a3a0 legacy=0x7f0e880c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.371+0000 7f0e86ffd700 1 -- 172.21.15.12:0/1727348037 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e880cbd10 con 0x7f0e8815e990 2023-07-08T12:39:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.371+0000 7f0e867fc700 1 -- 172.21.15.12:0/1727348037 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384160081 0 0) 0x7f0e78001110 con 0x7f0e8815a3a0 2023-07-08T12:39:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.371+0000 7f0e86ffd700 1 --2- 172.21.15.12:0/1727348037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e8815e990 0x7f0e88069f30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009840 tx=0x7f0e70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71447668c663b1c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.371+0000 7f0e867fc700 1 -- 172.21.15.12:0/1727348037 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70011830 con 0x7f0e8815e990 2023-07-08T12:39:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.371+0000 7f0e867fc700 1 -- 172.21.15.12:0/1727348037 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e70011990 con 0x7f0e8815e990 2023-07-08T12:39:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.371+0000 7f0e867fc700 1 -- 172.21.15.12:0/1727348037 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70011d20 con 0x7f0e8815e990 2023-07-08T12:39:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.371+0000 7f0e8db48700 1 -- 172.21.15.12:0/1727348037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e8815e990 msgr2=0x7f0e88069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.371+0000 7f0e8db48700 1 --2- 172.21.15.12:0/1727348037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e8815e990 0x7f0e88069f30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009840 tx=0x7f0e70009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.372+0000 7f0e8db48700 1 -- 172.21.15.12:0/1727348037 shutdown_connections 2023-07-08T12:39:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.372+0000 7f0e8db48700 1 --2- 172.21.15.12:0/1727348037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e8815e990 0x7f0e88069f30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.372+0000 7f0e8db48700 1 -- 172.21.15.12:0/1727348037 >> 172.21.15.12:0/1727348037 conn(0x7f0e880cc290 msgr2=0x7f0e880cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:19.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.372+0000 7f0e8db48700 1 -- 172.21.15.12:0/1727348037 shutdown_connections 2023-07-08T12:39:19.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.372+0000 7f0e8db48700 1 -- 172.21.15.12:0/1727348037 wait complete. 2023-07-08T12:39:19.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.373+0000 7f0e8db48700 1 Processor -- start 2023-07-08T12:39:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.373+0000 7f0e8db48700 1 -- start start 2023-07-08T12:39:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e8db48700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e8806a470 0x7f0e88158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e8db48700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e8815a3a0 0x7f0e880c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e8db48700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e8815e990 0x7f0e880c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e8db48700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880c4db0 con 0x7f0e8815e990 2023-07-08T12:39:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e8db48700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8815b670 con 0x7f0e8806a470 2023-07-08T12:39:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e8db48700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880c4230 con 0x7f0e8815a3a0 2023-07-08T12:39:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e87fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e8815e990 0x7f0e880c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e87fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e8815e990 0x7f0e880c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51376/0 (socket says 172.21.15.12:51376) 2023-07-08T12:39:19.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e87fff700 1 -- 172.21.15.12:0/3664172638 learned_addr learned my addr 172.21.15.12:0/3664172638 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:19.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e877fe700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e8806a470 0x7f0e88158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e86ffd700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e8815a3a0 0x7f0e880c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e87fff700 1 -- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e8815a3a0 msgr2=0x7f0e880c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e87fff700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e8815a3a0 0x7f0e880c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e87fff700 1 -- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e8806a470 msgr2=0x7f0e88158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e87fff700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e8806a470 0x7f0e88158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e87fff700 1 -- 172.21.15.12:0/3664172638 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e700094a0 con 0x7f0e8815e990 2023-07-08T12:39:19.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e86ffd700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e8815a3a0 0x7f0e880c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:19.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e877fe700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e8806a470 0x7f0e88158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:19.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.374+0000 7f0e87fff700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e8815e990 0x7f0e880c57c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000cc50 tx=0x7f0e8000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:19.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.375+0000 7f0e7ffff700 1 -- 172.21.15.12:0/3664172638 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000ad80 con 0x7f0e8815e990 2023-07-08T12:39:19.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.375+0000 7f0e7ffff700 1 -- 172.21.15.12:0/3664172638 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8000ce50 con 0x7f0e8815e990 2023-07-08T12:39:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.375+0000 7f0e8db48700 1 -- 172.21.15.12:0/3664172638 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e880c4510 con 0x7f0e8815e990 2023-07-08T12:39:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.375+0000 7f0e7ffff700 1 -- 172.21.15.12:0/3664172638 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80003c40 con 0x7f0e8815e990 2023-07-08T12:39:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.375+0000 7f0e8db48700 1 -- 172.21.15.12:0/3664172638 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e880c48a0 con 0x7f0e8815e990 2023-07-08T12:39:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.376+0000 7f0e8db48700 1 -- 172.21.15.12:0/3664172638 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8804ea60 con 0x7f0e8815e990 2023-07-08T12:39:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.376+0000 7f0e7ffff700 1 -- 172.21.15.12:0/3664172638 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e800050d0 con 0x7f0e8815e990 2023-07-08T12:39:19.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.377+0000 7f0e7ffff700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0e6806ace0 0x7f0e6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.377+0000 7f0e7ffff700 1 -- 172.21.15.12:0/3664172638 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0e80089880 con 0x7f0e8815e990 2023-07-08T12:39:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.377+0000 7f0e877fe700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0e6806ace0 0x7f0e6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.377+0000 7f0e877fe700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0e6806ace0 0x7f0e6806d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0e78007990 tx=0x7f0e78007b60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:19.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.381+0000 7f0e7ffff700 1 -- 172.21.15.12:0/3664172638 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0e80054600 con 0x7f0e8815e990 2023-07-08T12:39:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.627+0000 7f0e8db48700 1 -- 172.21.15.12:0/3664172638 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0e8806adc0 con 0x7f0e8815e990 2023-07-08T12:39:19.630 INFO:tasks.ceph.mon.a.smithi012.stderr:2023-07-08T12:39:19.628+0000 7f7a05dc0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-08T12:39:19.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.632+0000 7f0e7ffff700 1 -- 172.21.15.12:0/3664172638 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0e80057c20 con 0x7f0e8815e990 2023-07-08T12:39:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.635+0000 7f0e8db48700 1 -- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0e6806ace0 msgr2=0x7f0e6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.635+0000 7f0e8db48700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0e6806ace0 0x7f0e6806d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0e78007990 tx=0x7f0e78007b60 comp rx=0 tx=0).stop 2023-07-08T12:39:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.635+0000 7f0e8db48700 1 -- 172.21.15.12:0/3664172638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e8815e990 msgr2=0x7f0e880c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.635+0000 7f0e8db48700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e8815e990 0x7f0e880c57c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000cc50 tx=0x7f0e8000a340 comp rx=0 tx=0).stop 2023-07-08T12:39:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.635+0000 7f0e8db48700 1 -- 172.21.15.12:0/3664172638 shutdown_connections 2023-07-08T12:39:19.638 INFO:tasks.ceph.mds.a.smithi012.stderr: -5946> 2023-07-08T12:38:13.512+0000 7f2bd8677700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-08T12:39:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.635+0000 7f0e8db48700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f0e6806ace0 0x7f0e6806d190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.635+0000 7f0e8db48700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e8815e990 0x7f0e880c57c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.635+0000 7f0e8db48700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e8806a470 0x7f0e88158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.635+0000 7f0e8db48700 1 --2- 172.21.15.12:0/3664172638 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e8815a3a0 0x7f0e880c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.635+0000 7f0e8db48700 1 -- 172.21.15.12:0/3664172638 >> 172.21.15.12:0/3664172638 conn(0x7f0e880cc290 msgr2=0x7f0e8806f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:19.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.636+0000 7f0e8db48700 1 -- 172.21.15.12:0/3664172638 shutdown_connections 2023-07-08T12:39:19.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.636+0000 7f0e8db48700 1 -- 172.21.15.12:0/3664172638 wait complete. 2023-07-08T12:39:19.640 INFO:teuthology.orchestra.run.smithi012.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-08T12:39:19.651 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-08T12:39:19.669 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-07-08T12:39:19.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.776+0000 7fc8d8916700 1 Processor -- start 2023-07-08T12:39:19.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.776+0000 7fc8d8916700 1 -- start start 2023-07-08T12:39:19.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d8916700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc8d40cf3a0 0x7fc8d4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d8916700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc8d406a470 0x7fc8d406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d8916700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d4160ff0 con 0x7fc8d40cb920 2023-07-08T12:39:19.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d8916700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d4160bd0 con 0x7fc8d40cf3a0 2023-07-08T12:39:19.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d8916700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d40cbcf0 con 0x7fc8d406a470 2023-07-08T12:39:19.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d1d9b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc8d40cf3a0 0x7fc8d4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d259c700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc8d40cb920 0x7fc8d40cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38788/0 (socket says 172.21.15.12:38788) 2023-07-08T12:39:19.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d259c700 1 -- 172.21.15.12:0/1531205093 learned_addr learned my addr 172.21.15.12:0/1531205093 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:19.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d2d9d700 1 --2- 172.21.15.12:0/1531205093 >> v2:172.21.15.160:3300/0 conn(0x7fc8d406a470 0x7fc8d406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d1d9b700 1 -- 172.21.15.12:0/1531205093 >> v2:172.21.15.160:3300/0 conn(0x7fc8d406a470 msgr2=0x7fc8d406efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:19.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d1d9b700 1 --2- 172.21.15.12:0/1531205093 >> v2:172.21.15.160:3300/0 conn(0x7fc8d406a470 0x7fc8d406efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d1d9b700 1 -- 172.21.15.12:0/1531205093 >> v1:172.21.15.12:6789/0 conn(0x7fc8d40cb920 legacy=0x7fc8d40cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d159a700 1 -- 172.21.15.12:0/1531205093 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482227276 0 0) 0x7fc8d4160ff0 con 0x7fc8d40cb920 2023-07-08T12:39:19.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.777+0000 7fc8d1d9b700 1 -- 172.21.15.12:0/1531205093 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d406a8b0 con 0x7fc8d40cf3a0 2023-07-08T12:39:19.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.778+0000 7fc8d1d9b700 1 --2- 172.21.15.12:0/1531205093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d40cf3a0 0x7fc8d4069f30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40713a0 tx=0x7fc8c4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406b888b25f5e1db server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:19.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.778+0000 7fc8d159a700 1 -- 172.21.15.12:0/1531205093 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4011830 con 0x7fc8d40cf3a0 2023-07-08T12:39:19.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.778+0000 7fc8d159a700 1 -- 172.21.15.12:0/1531205093 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c4011990 con 0x7fc8d40cf3a0 2023-07-08T12:39:19.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.778+0000 7fc8d159a700 1 -- 172.21.15.12:0/1531205093 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4011d20 con 0x7fc8d40cf3a0 2023-07-08T12:39:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.778+0000 7fc8d8916700 1 -- 172.21.15.12:0/1531205093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d40cf3a0 msgr2=0x7fc8d4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.778+0000 7fc8d8916700 1 --2- 172.21.15.12:0/1531205093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d40cf3a0 0x7fc8d4069f30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40713a0 tx=0x7fc8c4009b20 comp rx=0 tx=0).stop 2023-07-08T12:39:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.778+0000 7fc8d8916700 1 -- 172.21.15.12:0/1531205093 shutdown_connections 2023-07-08T12:39:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.778+0000 7fc8d8916700 1 --2- 172.21.15.12:0/1531205093 >> v2:172.21.15.160:3300/0 conn(0x7fc8d406a470 0x7fc8d406efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.778+0000 7fc8d8916700 1 --2- 172.21.15.12:0/1531205093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d40cf3a0 0x7fc8d4069f30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.778+0000 7fc8d8916700 1 -- 172.21.15.12:0/1531205093 >> 172.21.15.12:0/1531205093 conn(0x7fc8d40cc390 msgr2=0x7fc8d40cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.778+0000 7fc8d8916700 1 -- 172.21.15.12:0/1531205093 shutdown_connections 2023-07-08T12:39:19.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.778+0000 7fc8d8916700 1 -- 172.21.15.12:0/1531205093 wait complete. 2023-07-08T12:39:19.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.780+0000 7fc8d8916700 1 Processor -- start 2023-07-08T12:39:19.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.780+0000 7fc8d8916700 1 -- start start 2023-07-08T12:39:19.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d8916700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d406a470 0x7fc8d40b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d8916700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8d40cb920 0x7fc8d40c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d8916700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8d40cf3a0 0x7fc8d40b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d8916700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d40af5e0 con 0x7fc8d406a470 2023-07-08T12:39:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d8916700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d40b27b0 con 0x7fc8d40cb920 2023-07-08T12:39:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d8916700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d40b2a90 con 0x7fc8d40cf3a0 2023-07-08T12:39:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d406a470 0x7fc8d40b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d406a470 0x7fc8d40b1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51398/0 (socket says 172.21.15.12:51398) 2023-07-08T12:39:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d259c700 1 -- 172.21.15.12:0/711423269 learned_addr learned my addr 172.21.15.12:0/711423269 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d1d9b700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8d40cb920 0x7fc8d40c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d2d9d700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8d40cf3a0 0x7fc8d40b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d259c700 1 -- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8d40cf3a0 msgr2=0x7fc8d40b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d259c700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8d40cf3a0 0x7fc8d40b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d259c700 1 -- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8d40cb920 msgr2=0x7fc8d40c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:19.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d259c700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8d40cb920 0x7fc8d40c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:19.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d259c700 1 -- 172.21.15.12:0/711423269 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c40094a0 con 0x7fc8d406a470 2023-07-08T12:39:19.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d259c700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d406a470 0x7fc8d40b1a30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80011e0 tx=0x7fc8c800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:19.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8c2ffd700 1 -- 172.21.15.12:0/711423269 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8004c80 con 0x7fc8d406a470 2023-07-08T12:39:19.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8c2ffd700 1 -- 172.21.15.12:0/711423269 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c8004e00 con 0x7fc8d406a470 2023-07-08T12:39:19.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8c2ffd700 1 -- 172.21.15.12:0/711423269 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8007310 con 0x7fc8d406a470 2023-07-08T12:39:19.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.781+0000 7fc8d8916700 1 -- 172.21.15.12:0/711423269 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d40b2d70 con 0x7fc8d406a470 2023-07-08T12:39:19.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.782+0000 7fc8d8916700 1 -- 172.21.15.12:0/711423269 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d40b3330 con 0x7fc8d406a470 2023-07-08T12:39:19.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.783+0000 7fc8d8916700 1 -- 172.21.15.12:0/711423269 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d406bfd0 con 0x7fc8d406a470 2023-07-08T12:39:19.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.783+0000 7fc8c2ffd700 1 -- 172.21.15.12:0/711423269 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc8c8007470 con 0x7fc8d406a470 2023-07-08T12:39:19.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.783+0000 7fc8c2ffd700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc8b406ace0 0x7fc8b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:19.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.783+0000 7fc8c2ffd700 1 -- 172.21.15.12:0/711423269 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fc8c80895a0 con 0x7fc8d406a470 2023-07-08T12:39:19.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.784+0000 7fc8d1d9b700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc8b406ace0 0x7fc8b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:19.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.784+0000 7fc8d1d9b700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc8b406ace0 0x7fc8b406d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40b5810 tx=0x7fc8c4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:19.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:19.788+0000 7fc8c2ffd700 1 -- 172.21.15.12:0/711423269 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc8c80542d0 con 0x7fc8d406a470 2023-07-08T12:39:20.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.031+0000 7fc8d8916700 1 -- 172.21.15.12:0/711423269 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc8d40b3550 con 0x7fc8d406a470 2023-07-08T12:39:20.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.638+0000 7fc8c2ffd700 1 -- 172.21.15.12:0/711423269 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc8c80578f0 con 0x7fc8d406a470 2023-07-08T12:39:20.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.641+0000 7fc8d8916700 1 -- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc8b406ace0 msgr2=0x7fc8b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:20.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.641+0000 7fc8d8916700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc8b406ace0 0x7fc8b406d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40b5810 tx=0x7fc8c4008030 comp rx=0 tx=0).stop 2023-07-08T12:39:20.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.641+0000 7fc8d8916700 1 -- 172.21.15.12:0/711423269 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d406a470 msgr2=0x7fc8d40b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:20.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.641+0000 7fc8d8916700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d406a470 0x7fc8d40b1a30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80011e0 tx=0x7fc8c800cae0 comp rx=0 tx=0).stop 2023-07-08T12:39:20.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.641+0000 7fc8d8916700 1 -- 172.21.15.12:0/711423269 shutdown_connections 2023-07-08T12:39:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.641+0000 7fc8d8916700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fc8b406ace0 0x7fc8b406d190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.641+0000 7fc8d8916700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8d40cf3a0 0x7fc8d40b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.641+0000 7fc8d8916700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8d406a470 0x7fc8d40b1a30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.641+0000 7fc8d8916700 1 --2- 172.21.15.12:0/711423269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8d40cb920 0x7fc8d40c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.641+0000 7fc8d8916700 1 -- 172.21.15.12:0/711423269 >> 172.21.15.12:0/711423269 conn(0x7fc8d40cc390 msgr2=0x7fc8d415cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.641+0000 7fc8d8916700 1 -- 172.21.15.12:0/711423269 shutdown_connections 2023-07-08T12:39:20.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.641+0000 7fc8d8916700 1 -- 172.21.15.12:0/711423269 wait complete. 2023-07-08T12:39:20.660 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-08T12:39:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.836+0000 7f70ab86b700 1 Processor -- start 2023-07-08T12:39:20.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.837+0000 7f70ab86b700 1 -- start start 2023-07-08T12:39:20.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.837+0000 7f70ab86b700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f70a40ce9a0 0x7f70a415caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:20.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.837+0000 7f70ab86b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70a40ccfa0 0x7f70a40cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:20.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.837+0000 7f70ab86b700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a415cfe0 con 0x7f70a40cd980 2023-07-08T12:39:20.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.837+0000 7f70ab86b700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a40c4320 con 0x7f70a40ccfa0 2023-07-08T12:39:20.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f70ab86b700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a40cdda0 con 0x7f70a40ce9a0 2023-07-08T12:39:20.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f70a8e06700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70a40ccfa0 0x7f70a40cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:20.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f70a8e06700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70a40ccfa0 0x7f70a40cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51408/0 (socket says 172.21.15.12:51408) 2023-07-08T12:39:20.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f70a8e06700 1 -- 172.21.15.12:0/278190102 learned_addr learned my addr 172.21.15.12:0/278190102 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:20.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f70a9607700 1 --2- 172.21.15.12:0/278190102 >> v2:172.21.15.160:3300/0 conn(0x7f70a40ce9a0 0x7f70a415caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:20.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f70a8e06700 1 -- 172.21.15.12:0/278190102 >> v2:172.21.15.160:3300/0 conn(0x7f70a40ce9a0 msgr2=0x7f70a415caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:20.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f70a8e06700 1 --2- 172.21.15.12:0/278190102 >> v2:172.21.15.160:3300/0 conn(0x7f70a40ce9a0 0x7f70a415caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:20.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f70a8e06700 1 -- 172.21.15.12:0/278190102 >> v1:172.21.15.160:6789/0 conn(0x7f70a40cd980 legacy=0x7f70a40b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:20.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f70a8e06700 1 -- 172.21.15.12:0/278190102 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a406b270 con 0x7f70a40ccfa0 2023-07-08T12:39:20.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f70a9607700 1 --2- 172.21.15.12:0/278190102 >> v2:172.21.15.160:3300/0 conn(0x7f70a40ce9a0 0x7f70a415caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:20.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f70a8e06700 1 --2- 172.21.15.12:0/278190102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a40ccfa0 0x7f70a40cd440 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f70a406fb50 tx=0x7f7098009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb98d60c0fef3f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:20.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f7097fff700 1 -- 172.21.15.12:0/278190102 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098004230 con 0x7f70a40ccfa0 2023-07-08T12:39:20.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f7097fff700 1 -- 172.21.15.12:0/278190102 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7098004390 con 0x7f70a40ccfa0 2023-07-08T12:39:20.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.838+0000 7f7097fff700 1 -- 172.21.15.12:0/278190102 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70980036a0 con 0x7f70a40ccfa0 2023-07-08T12:39:20.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.839+0000 7f70ab86b700 1 -- 172.21.15.12:0/278190102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a40ccfa0 msgr2=0x7f70a40cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:20.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.839+0000 7f70ab86b700 1 --2- 172.21.15.12:0/278190102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a40ccfa0 0x7f70a40cd440 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f70a406fb50 tx=0x7f7098009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:20.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.839+0000 7f70ab86b700 1 -- 172.21.15.12:0/278190102 shutdown_connections 2023-07-08T12:39:20.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.839+0000 7f70ab86b700 1 --2- 172.21.15.12:0/278190102 >> v2:172.21.15.160:3300/0 conn(0x7f70a40ce9a0 0x7f70a415caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:20.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.839+0000 7f70ab86b700 1 --2- 172.21.15.12:0/278190102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a40ccfa0 0x7f70a40cd440 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:20.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.839+0000 7f70ab86b700 1 -- 172.21.15.12:0/278190102 >> 172.21.15.12:0/278190102 conn(0x7f70a406e670 msgr2=0x7f70a406ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:20.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.839+0000 7f70ab86b700 1 -- 172.21.15.12:0/278190102 shutdown_connections 2023-07-08T12:39:20.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.839+0000 7f70ab86b700 1 -- 172.21.15.12:0/278190102 wait complete. 2023-07-08T12:39:20.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.840+0000 7f70ab86b700 1 Processor -- start 2023-07-08T12:39:20.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.840+0000 7f70ab86b700 1 -- start start 2023-07-08T12:39:20.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70ab86b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a40ccfa0 0x7f70a4152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:20.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70ab86b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70a40cd980 0x7f70a40b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:20.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70ab86b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a40ce9a0 0x7f70a41f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:20.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70ab86b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a415dab0 con 0x7f70a40ccfa0 2023-07-08T12:39:20.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70ab86b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a41532c0 con 0x7f70a40ce9a0 2023-07-08T12:39:20.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70ab86b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a41f3540 con 0x7f70a40cd980 2023-07-08T12:39:20.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70a9607700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a40ccfa0 0x7f70a4152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:20.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70a9607700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a40ccfa0 0x7f70a4152d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51414/0 (socket says 172.21.15.12:51414) 2023-07-08T12:39:20.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70a9607700 1 -- 172.21.15.12:0/2404209362 learned_addr learned my addr 172.21.15.12:0/2404209362 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:20.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70a9e08700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a40ce9a0 0x7f70a41f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:20.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70a8e06700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70a40cd980 0x7f70a40b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:20.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70a9607700 1 -- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70a40cd980 msgr2=0x7f70a40b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:20.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70a9607700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70a40cd980 0x7f70a40b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:20.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70a9607700 1 -- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a40ce9a0 msgr2=0x7f70a41f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:20.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70a9607700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a40ce9a0 0x7f70a41f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:20.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.841+0000 7f70a9607700 1 -- 172.21.15.12:0/2404209362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7098009550 con 0x7f70a40ccfa0 2023-07-08T12:39:20.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.842+0000 7f70a9607700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a40ccfa0 0x7f70a4152d80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f70a000c750 tx=0x7f70a000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:20.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.842+0000 7f7095ffb700 1 -- 172.21.15.12:0/2404209362 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0011800 con 0x7f70a40ccfa0 2023-07-08T12:39:20.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.842+0000 7f7095ffb700 1 -- 172.21.15.12:0/2404209362 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a0011960 con 0x7f70a40ccfa0 2023-07-08T12:39:20.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.842+0000 7f7095ffb700 1 -- 172.21.15.12:0/2404209362 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0011cd0 con 0x7f70a40ccfa0 2023-07-08T12:39:20.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.842+0000 7f70ab86b700 1 -- 172.21.15.12:0/2404209362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a41f4c80 con 0x7f70a40ccfa0 2023-07-08T12:39:20.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.842+0000 7f70ab86b700 1 -- 172.21.15.12:0/2404209362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a40b5e10 con 0x7f70a40ccfa0 2023-07-08T12:39:20.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.843+0000 7f70ab86b700 1 -- 172.21.15.12:0/2404209362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a4074180 con 0x7f70a40ccfa0 2023-07-08T12:39:20.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.843+0000 7f7095ffb700 1 -- 172.21.15.12:0/2404209362 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70a0021350 con 0x7f70a40ccfa0 2023-07-08T12:39:20.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.844+0000 7f7095ffb700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f708806ace0 0x7f708806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:20.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.844+0000 7f7095ffb700 1 -- 172.21.15.12:0/2404209362 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f70a0089800 con 0x7f70a40ccfa0 2023-07-08T12:39:20.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.844+0000 7f70a8e06700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f708806ace0 0x7f708806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:20.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.845+0000 7f70a8e06700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f708806ace0 0x7f708806d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7098009130 tx=0x7f7098008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:20.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:20.849+0000 7f7095ffb700 1 -- 172.21.15.12:0/2404209362 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f70a0054530 con 0x7f70a40ccfa0 2023-07-08T12:39:21.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.047+0000 7f70ab86b700 1 -- 172.21.15.12:0/2404209362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f70a4071030 con 0x7f70a40ccfa0 2023-07-08T12:39:21.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.641+0000 7f7095ffb700 1 -- 172.21.15.12:0/2404209362 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v32) v1 ==== 181+0+0 (secure 0 0 0) 0x7f70a0057b50 con 0x7f70a40ccfa0 2023-07-08T12:39:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.646+0000 7f70ab86b700 1 -- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f708806ace0 msgr2=0x7f708806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.646+0000 7f70ab86b700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f708806ace0 0x7f708806d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7098009130 tx=0x7f7098008030 comp rx=0 tx=0).stop 2023-07-08T12:39:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.646+0000 7f70ab86b700 1 -- 172.21.15.12:0/2404209362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a40ccfa0 msgr2=0x7f70a4152d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.646+0000 7f70ab86b700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a40ccfa0 0x7f70a4152d80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f70a000c750 tx=0x7f70a000cb10 comp rx=0 tx=0).stop 2023-07-08T12:39:21.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.646+0000 7f70ab86b700 1 -- 172.21.15.12:0/2404209362 shutdown_connections 2023-07-08T12:39:21.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.646+0000 7f70ab86b700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f708806ace0 0x7f708806d190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:21.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.646+0000 7f70ab86b700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a40ce9a0 0x7f70a41f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:21.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.646+0000 7f70ab86b700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70a40ccfa0 0x7f70a4152d80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:21.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.646+0000 7f70ab86b700 1 --2- 172.21.15.12:0/2404209362 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70a40cd980 0x7f70a40b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:21.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.646+0000 7f70ab86b700 1 -- 172.21.15.12:0/2404209362 >> 172.21.15.12:0/2404209362 conn(0x7f70a406e670 msgr2=0x7f70a415dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:21.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.646+0000 7f70ab86b700 1 -- 172.21.15.12:0/2404209362 shutdown_connections 2023-07-08T12:39:21.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.647+0000 7f70ab86b700 1 -- 172.21.15.12:0/2404209362 wait complete. 2023-07-08T12:39:21.650 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' removed 2023-07-08T12:39:21.664 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-08T12:39:21.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.798+0000 7f132ec09700 1 Processor -- start 2023-07-08T12:39:21.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.798+0000 7f132ec09700 1 -- start start 2023-07-08T12:39:21.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132ec09700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132815a2e0 0x7f13280ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:21.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132ec09700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f13280cb490 0x7f1328070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:21.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132ec09700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13280ce840 con 0x7f132815a6b0 2023-07-08T12:39:21.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132ec09700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13280c2990 con 0x7f13280cb490 2023-07-08T12:39:21.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132ec09700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13280cb8e0 con 0x7f132815a2e0 2023-07-08T12:39:21.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132c9a5700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132815a2e0 0x7f13280ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:21.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132d1a6700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f13280cb490 0x7f1328070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:21.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132d1a6700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f13280cb490 0x7f1328070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50170/0 (socket says 172.21.15.12:50170) 2023-07-08T12:39:21.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132d1a6700 1 -- 172.21.15.12:0/1988613619 learned_addr learned my addr 172.21.15.12:0/1988613619 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:21.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132d1a6700 1 -- 172.21.15.12:0/1988613619 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132815a2e0 msgr2=0x7f13280ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:21.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132d1a6700 1 --2- 172.21.15.12:0/1988613619 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132815a2e0 0x7f13280ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:21.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132d1a6700 1 -- 172.21.15.12:0/1988613619 >> v1:172.21.15.160:6789/0 conn(0x7f132815a6b0 legacy=0x7f13280ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:21.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132d1a6700 1 -- 172.21.15.12:0/1988613619 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13280cd090 con 0x7f13280cb490 2023-07-08T12:39:21.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.799+0000 7f132c9a5700 1 --2- 172.21.15.12:0/1988613619 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132815a2e0 0x7f13280ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:39:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.800+0000 7f132d1a6700 1 --2- 172.21.15.12:0/1988613619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13280cb490 0x7f1328070d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1320009e50 tx=0x7f132000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78abc57247b26e9f server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.800+0000 7f13277fe700 1 -- 172.21.15.12:0/1988613619 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132000e070 con 0x7f13280cb490 2023-07-08T12:39:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.800+0000 7f13277fe700 1 -- 172.21.15.12:0/1988613619 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132000b910 con 0x7f13280cb490 2023-07-08T12:39:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.800+0000 7f13277fe700 1 -- 172.21.15.12:0/1988613619 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132000bcc0 con 0x7f13280cb490 2023-07-08T12:39:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.800+0000 7f132ec09700 1 -- 172.21.15.12:0/1988613619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13280cb490 msgr2=0x7f1328070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.800+0000 7f132ec09700 1 --2- 172.21.15.12:0/1988613619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13280cb490 0x7f1328070d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1320009e50 tx=0x7f132000b330 comp rx=0 tx=0).stop 2023-07-08T12:39:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.800+0000 7f132ec09700 1 -- 172.21.15.12:0/1988613619 shutdown_connections 2023-07-08T12:39:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.800+0000 7f132ec09700 1 --2- 172.21.15.12:0/1988613619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13280cb490 0x7f1328070d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.800+0000 7f132ec09700 1 --2- 172.21.15.12:0/1988613619 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132815a2e0 0x7f13280ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.800+0000 7f132ec09700 1 -- 172.21.15.12:0/1988613619 >> 172.21.15.12:0/1988613619 conn(0x7f13280cf060 msgr2=0x7f13280cf460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.800+0000 7f132ec09700 1 -- 172.21.15.12:0/1988613619 shutdown_connections 2023-07-08T12:39:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.800+0000 7f132ec09700 1 -- 172.21.15.12:0/1988613619 wait complete. 2023-07-08T12:39:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.802+0000 7f132ec09700 1 Processor -- start 2023-07-08T12:39:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.802+0000 7f132ec09700 1 -- start start 2023-07-08T12:39:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.802+0000 7f132ec09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13280cb490 0x7f1328154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.802+0000 7f132ec09700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132815a2e0 0x7f1328154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.802+0000 7f132ec09700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f132815a6b0 0x7f13280c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.802+0000 7f132ec09700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132815bfd0 con 0x7f13280cb490 2023-07-08T12:39:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132ec09700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13280c3ab0 con 0x7f132815a6b0 2023-07-08T12:39:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132ec09700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13280c3d30 con 0x7f132815a2e0 2023-07-08T12:39:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132c9a5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13280cb490 0x7f1328154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132c9a5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13280cb490 0x7f1328154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51428/0 (socket says 172.21.15.12:51428) 2023-07-08T12:39:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132c9a5700 1 -- 172.21.15.12:0/1130893489 learned_addr learned my addr 172.21.15.12:0/1130893489 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132d1a6700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f132815a6b0 0x7f13280c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:21.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f1327fff700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132815a2e0 0x7f1328154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:21.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132c9a5700 1 -- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132815a2e0 msgr2=0x7f1328154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:21.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132c9a5700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132815a2e0 0x7f1328154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:21.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132c9a5700 1 -- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f132815a6b0 msgr2=0x7f13280c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:21.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132c9a5700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f132815a6b0 0x7f13280c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:21.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132c9a5700 1 -- 172.21.15.12:0/1130893489 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c0094a0 con 0x7f13280cb490 2023-07-08T12:39:21.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132d1a6700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f132815a6b0 0x7f13280c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:21.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132c9a5700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13280cb490 0x7f1328154ab0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f131c00ebe0 tx=0x7f131c009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:21.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f13257fa700 1 -- 172.21.15.12:0/1130893489 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c00c8b0 con 0x7f13280cb490 2023-07-08T12:39:21.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132ec09700 1 -- 172.21.15.12:0/1130893489 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1320009b30 con 0x7f13280cb490 2023-07-08T12:39:21.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f13257fa700 1 -- 172.21.15.12:0/1130893489 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131c00ca10 con 0x7f13280cb490 2023-07-08T12:39:21.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f13257fa700 1 -- 172.21.15.12:0/1130893489 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c00cda0 con 0x7f13280cb490 2023-07-08T12:39:21.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.803+0000 7f132ec09700 1 -- 172.21.15.12:0/1130893489 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13280c42f0 con 0x7f13280cb490 2023-07-08T12:39:21.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.804+0000 7f132ec09700 1 -- 172.21.15.12:0/1130893489 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13280b5070 con 0x7f13280cb490 2023-07-08T12:39:21.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.805+0000 7f13257fa700 1 -- 172.21.15.12:0/1130893489 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f131c013470 con 0x7f13280cb490 2023-07-08T12:39:21.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.805+0000 7f13257fa700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f130c06ace0 0x7f130c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:21.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.805+0000 7f13257fa700 1 -- 172.21.15.12:0/1130893489 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f131c089800 con 0x7f13280cb490 2023-07-08T12:39:21.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.805+0000 7f1327fff700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f130c06ace0 0x7f130c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:21.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.806+0000 7f1327fff700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f130c06ace0 0x7f130c06d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1328159cf0 tx=0x7f1314009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:21.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:21.810+0000 7f13257fa700 1 -- 172.21.15.12:0/1130893489 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f131c0546e0 con 0x7f13280cb490 2023-07-08T12:39:22.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.009+0000 7f132ec09700 1 -- 172.21.15.12:0/1130893489 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f132815c590 con 0x7f13280cb490 2023-07-08T12:39:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.654+0000 7f13257fa700 1 -- 172.21.15.12:0/1130893489 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v33) v1 ==== 169+0+0 (secure 0 0 0) 0x7f131c057d00 con 0x7f13280cb490 2023-07-08T12:39:22.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.665+0000 7f132ec09700 1 -- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f130c06ace0 msgr2=0x7f130c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:22.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.665+0000 7f132ec09700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f130c06ace0 0x7f130c06d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1328159cf0 tx=0x7f1314009040 comp rx=0 tx=0).stop 2023-07-08T12:39:22.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.665+0000 7f132ec09700 1 -- 172.21.15.12:0/1130893489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13280cb490 msgr2=0x7f1328154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:22.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.665+0000 7f132ec09700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13280cb490 0x7f1328154ab0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f131c00ebe0 tx=0x7f131c009e20 comp rx=0 tx=0).stop 2023-07-08T12:39:22.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.665+0000 7f132ec09700 1 -- 172.21.15.12:0/1130893489 shutdown_connections 2023-07-08T12:39:22.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.665+0000 7f132ec09700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f130c06ace0 0x7f130c06d190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:22.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.665+0000 7f132ec09700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f132815a6b0 0x7f13280c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:22.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.665+0000 7f132ec09700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13280cb490 0x7f1328154ab0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:22.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.665+0000 7f132ec09700 1 --2- 172.21.15.12:0/1130893489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132815a2e0 0x7f1328154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:22.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.665+0000 7f132ec09700 1 -- 172.21.15.12:0/1130893489 >> 172.21.15.12:0/1130893489 conn(0x7f13280cf060 msgr2=0x7f13280af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:22.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.665+0000 7f132ec09700 1 -- 172.21.15.12:0/1130893489 shutdown_connections 2023-07-08T12:39:22.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.666+0000 7f132ec09700 1 -- 172.21.15.12:0/1130893489 wait complete. 2023-07-08T12:39:22.670 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' removed 2023-07-08T12:39:22.685 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-07-08T12:39:22.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.821+0000 7f6e0b024700 1 Processor -- start 2023-07-08T12:39:22.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.821+0000 7f6e0b024700 1 -- start start 2023-07-08T12:39:22.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.822+0000 7f6e0b024700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0415a350 0x7f6e040b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:22.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.822+0000 7f6e0b024700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e040cdb30 con 0x7f6e040cde30 2023-07-08T12:39:22.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.822+0000 7f6e0b024700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e040cd850 con 0x7f6e040cf8a0 2023-07-08T12:39:22.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.822+0000 7f6e0b024700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04159870 con 0x7f6e0415a350 2023-07-08T12:39:22.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.822+0000 7f6e03fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6e040cde30 0x7f6e040715c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38834/0 (socket says 172.21.15.12:38834) 2023-07-08T12:39:22.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.822+0000 7f6e03fff700 1 -- 172.21.15.12:0/103165982 learned_addr learned my addr 172.21.15.12:0/103165982 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:22.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.822+0000 7f6e095c1700 1 --2- 172.21.15.12:0/103165982 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0415a350 0x7f6e040b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:22.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.822+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518174662 0 0) 0x7f6e040cdb30 con 0x7f6e040cde30 2023-07-08T12:39:22.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6df40035b0 con 0x7f6e040cde30 2023-07-08T12:39:22.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 958417903 0 0) 0x7f6df40035b0 con 0x7f6e040cde30 2023-07-08T12:39:22.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0415a350 msgr2=0x7f6e040b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:22.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 --2- 172.21.15.12:0/103165982 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0415a350 0x7f6e040b1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:22.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 >> v1:172.21.15.160:6789/0 conn(0x7f6e040cf8a0 legacy=0x7f6e040cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:22.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e04159b50 con 0x7f6e040cde30 2023-07-08T12:39:22.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f6df0001670 con 0x7f6e040cde30 2023-07-08T12:39:22.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 >> v1:172.21.15.12:6789/0 conn(0x7f6e040cde30 legacy=0x7f6e040715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:22.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 --2- 172.21.15.12:0/103165982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df4003a30 0x7f6df4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:22.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 --2- 172.21.15.12:0/103165982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6df4004850 0x7f6df4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:22.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 --2- 172.21.15.12:0/103165982 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0415a350 0x7f6df4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:22.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04159870 con 0x7f6df4003a30 2023-07-08T12:39:22.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df400bac0 con 0x7f6df4004850 2023-07-08T12:39:22.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df400bc50 con 0x7f6e0415a350 2023-07-08T12:39:22.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e08dc0700 1 --2- 172.21.15.12:0/103165982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df4003a30 0x7f6df4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:22.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e03fff700 1 --2- 172.21.15.12:0/103165982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6df4004850 0x7f6df4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:22.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e095c1700 1 --2- 172.21.15.12:0/103165982 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0415a350 0x7f6df4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:22.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e08dc0700 1 -- 172.21.15.12:0/103165982 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0415a350 msgr2=0x7f6df4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:22.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e08dc0700 1 --2- 172.21.15.12:0/103165982 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0415a350 0x7f6df4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:22.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e08dc0700 1 -- 172.21.15.12:0/103165982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6df4004850 msgr2=0x7f6df4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:22.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e08dc0700 1 --2- 172.21.15.12:0/103165982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6df4004850 0x7f6df4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:22.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e08dc0700 1 -- 172.21.15.12:0/103165982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df40035b0 con 0x7f6df4003a30 2023-07-08T12:39:22.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.823+0000 7f6e08dc0700 1 --2- 172.21.15.12:0/103165982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df4003a30 0x7f6df4003e40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6df80017c0 tx=0x7f6df4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110877b5c1b50fbc server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:22.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.824+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df80052d0 con 0x7f6df4003a30 2023-07-08T12:39:22.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.824+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df8005430 con 0x7f6df4003a30 2023-07-08T12:39:22.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.824+0000 7f6e037fe700 1 -- 172.21.15.12:0/103165982 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df80038c0 con 0x7f6df4003a30 2023-07-08T12:39:22.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.824+0000 7f6e0b024700 1 -- 172.21.15.12:0/103165982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df4003a30 msgr2=0x7f6df4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:22.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.824+0000 7f6e0b024700 1 --2- 172.21.15.12:0/103165982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df4003a30 0x7f6df4003e40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6df80017c0 tx=0x7f6df4005880 comp rx=0 tx=0).stop 2023-07-08T12:39:22.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.824+0000 7f6e095c1700 1 -- 172.21.15.12:0/103165982 reap_dead start 2023-07-08T12:39:22.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.824+0000 7f6e0b024700 1 -- 172.21.15.12:0/103165982 shutdown_connections 2023-07-08T12:39:22.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.824+0000 7f6e0b024700 1 -- 172.21.15.12:0/103165982 >> 172.21.15.12:0/103165982 conn(0x7f6e0406fd80 msgr2=0x7f6e040b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:22.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.824+0000 7f6e0b024700 1 -- 172.21.15.12:0/103165982 shutdown_connections 2023-07-08T12:39:22.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.824+0000 7f6e0b024700 1 -- 172.21.15.12:0/103165982 wait complete. 2023-07-08T12:39:22.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.825+0000 7f6e0b024700 1 Processor -- start 2023-07-08T12:39:22.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.825+0000 7f6e0b024700 1 -- start start 2023-07-08T12:39:22.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.826+0000 7f6e0b024700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df4003a30 0x7f6e0414e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:22.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.826+0000 7f6e0b024700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e04156470 0x7f6e040b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:22.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.826+0000 7f6e0b024700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0414ecb0 0x7f6e040b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:22.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.826+0000 7f6e0b024700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e041e63a0 con 0x7f6df4003a30 2023-07-08T12:39:22.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.826+0000 7f6e0b024700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04156900 con 0x7f6e04156470 2023-07-08T12:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.826+0000 7f6e0b024700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04156be0 con 0x7f6e0414ecb0 2023-07-08T12:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.826+0000 7f6e08dc0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df4003a30 0x7f6e0414e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.826+0000 7f6e08dc0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df4003a30 0x7f6e0414e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51448/0 (socket says 172.21.15.12:51448) 2023-07-08T12:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.826+0000 7f6e08dc0700 1 -- 172.21.15.12:0/2819799964 learned_addr learned my addr 172.21.15.12:0/2819799964 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e095c1700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0414ecb0 0x7f6e040b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e03fff700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e04156470 0x7f6e040b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:22.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e08dc0700 1 -- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0414ecb0 msgr2=0x7f6e040b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e08dc0700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0414ecb0 0x7f6e040b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e08dc0700 1 -- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e04156470 msgr2=0x7f6e040b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e08dc0700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e04156470 0x7f6e040b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e08dc0700 1 -- 172.21.15.12:0/2819799964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e04156ec0 con 0x7f6df4003a30 2023-07-08T12:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e03fff700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e04156470 0x7f6e040b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e095c1700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0414ecb0 0x7f6e040b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e08dc0700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df4003a30 0x7f6e0414e720 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6df40047a0 tx=0x7f6df8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:22.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e017fa700 1 -- 172.21.15.12:0/2819799964 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8007240 con 0x7f6df4003a30 2023-07-08T12:39:22.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e017fa700 1 -- 172.21.15.12:0/2819799964 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df80073a0 con 0x7f6df4003a30 2023-07-08T12:39:22.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e0b024700 1 -- 172.21.15.12:0/2819799964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e041571a0 con 0x7f6df4003a30 2023-07-08T12:39:22.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e017fa700 1 -- 172.21.15.12:0/2819799964 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8001290 con 0x7f6df4003a30 2023-07-08T12:39:22.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.827+0000 7f6e0b024700 1 -- 172.21.15.12:0/2819799964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e040b5490 con 0x7f6df4003a30 2023-07-08T12:39:22.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.828+0000 7f6e0b024700 1 -- 172.21.15.12:0/2819799964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e041576c0 con 0x7f6df4003a30 2023-07-08T12:39:22.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.828+0000 7f6e017fa700 1 -- 172.21.15.12:0/2819799964 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6df8003680 con 0x7f6df4003a30 2023-07-08T12:39:22.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.829+0000 7f6e017fa700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6de806ad10 0x7f6de806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.829+0000 7f6e017fa700 1 -- 172.21.15.12:0/2819799964 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6047+0+0 (secure 0 0 0) 0x7f6df808a3d0 con 0x7f6df4003a30 2023-07-08T12:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.829+0000 7f6e03fff700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6de806ad10 0x7f6de806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.830+0000 7f6e03fff700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6de806ad10 0x7f6de806d1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6e0414ec60 tx=0x7f6df0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:22.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:22.833+0000 7f6e017fa700 1 -- 172.21.15.12:0/2819799964 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6e041576c0 con 0x7f6df4003a30 2023-07-08T12:39:23.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.032+0000 7f6e0b024700 1 -- 172.21.15.12:0/2819799964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e041576c0 con 0x7f6df4003a30 2023-07-08T12:39:23.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.659+0000 7f6e017fa700 1 -- 172.21.15.12:0/2819799964 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v34) v1 ==== 178+0+0 (secure 0 0 0) 0x7f6e041576c0 con 0x7f6df4003a30 2023-07-08T12:39:23.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.663+0000 7f6e0b024700 1 -- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6de806ad10 msgr2=0x7f6de806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:23.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.663+0000 7f6e0b024700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6de806ad10 0x7f6de806d1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6e0414ec60 tx=0x7f6df0009040 comp rx=0 tx=0).stop 2023-07-08T12:39:23.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.663+0000 7f6e0b024700 1 -- 172.21.15.12:0/2819799964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df4003a30 msgr2=0x7f6e0414e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:23.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.663+0000 7f6e0b024700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df4003a30 0x7f6e0414e720 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6df40047a0 tx=0x7f6df8001120 comp rx=0 tx=0).stop 2023-07-08T12:39:23.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.663+0000 7f6e0b024700 1 -- 172.21.15.12:0/2819799964 shutdown_connections 2023-07-08T12:39:23.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.663+0000 7f6e0b024700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f6de806ad10 0x7f6de806d1c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:23.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.663+0000 7f6e0b024700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0414ecb0 0x7f6e040b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:23.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.663+0000 7f6e0b024700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6df4003a30 0x7f6e0414e720 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:23.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.663+0000 7f6e0b024700 1 --2- 172.21.15.12:0/2819799964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e04156470 0x7f6e040b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:23.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.663+0000 7f6e0b024700 1 -- 172.21.15.12:0/2819799964 >> 172.21.15.12:0/2819799964 conn(0x7f6e0406fd80 msgr2=0x7f6e040cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:23.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.664+0000 7f6e0b024700 1 -- 172.21.15.12:0/2819799964 shutdown_connections 2023-07-08T12:39:23.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.664+0000 7f6e0b024700 1 -- 172.21.15.12:0/2819799964 wait complete. 2023-07-08T12:39:23.668 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data_ec' removed 2023-07-08T12:39:23.681 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-08T12:39:23.682 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-08T12:39:23.682 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-08T12:39:23.682 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T12:39:23.683 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-08T12:39:23.683 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-08T12:39:23.683 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-08T12:39:23.683 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T12:39:23.683 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-08T12:39:23.684 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-08T12:39:23.684 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-08T12:39:23.684 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T12:39:23.684 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-08T12:39:23.684 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-08T12:39:23.685 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-08T12:39:23.685 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T12:39:23.693 INFO:tasks.ceph.mds.a:Stopped 2023-07-08T12:39:23.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-08T12:39:23.694 INFO:tasks.ceph.mds.c:Stopped 2023-07-08T12:39:23.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-08T12:39:23.696 INFO:tasks.ceph.mds.b:Stopped 2023-07-08T12:39:23.696 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-08T12:39:23.697 INFO:tasks.ceph.mds.d:Stopped 2023-07-08T12:39:23.697 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-08T12:39:23.736 INFO:tasks.ceph.mds.a:Started 2023-07-08T12:39:23.737 DEBUG:teuthology.parallel:result is None 2023-07-08T12:39:23.739 INFO:tasks.ceph.mds.c:Started 2023-07-08T12:39:23.740 DEBUG:teuthology.parallel:result is None 2023-07-08T12:39:23.741 INFO:tasks.ceph.mds.b:Started 2023-07-08T12:39:23.742 INFO:tasks.ceph.mds.d:Started 2023-07-08T12:39:23.742 DEBUG:teuthology.parallel:result is None 2023-07-08T12:39:23.743 DEBUG:teuthology.parallel:result is None 2023-07-08T12:39:23.744 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-08T12:39:23.824 INFO:tasks.ceph.mds.c.smithi012.stdout:starting mds.c at 2023-07-08T12:39:23.824 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-07-08T12:39:23.827 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-07-08T12:39:23.827 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-07-08T12:39:23.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.885+0000 7f47e4fcd700 1 Processor -- start 2023-07-08T12:39:23.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.885+0000 7f47e4fcd700 1 -- start start 2023-07-08T12:39:23.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47e4fcd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47e00d2640 0x7f47e00b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:23.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47e4fcd700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e00d1790 con 0x7f47e015c460 2023-07-08T12:39:23.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47e4fcd700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e00cd950 con 0x7f47e00cf8f0 2023-07-08T12:39:23.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47e4fcd700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e00cedd0 con 0x7f47e00d2640 2023-07-08T12:39:23.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47de59c700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f47e015c460 0x7f47e00cf1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38878/0 (socket says 172.21.15.12:38878) 2023-07-08T12:39:23.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47de59c700 1 -- 172.21.15.12:0/3564303726 learned_addr learned my addr 172.21.15.12:0/3564303726 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:23.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47ddd9b700 1 --2- 172.21.15.12:0/3564303726 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47e00d2640 0x7f47e00b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:23.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47dd59a700 1 -- 172.21.15.12:0/3564303726 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873118300 0 0) 0x7f47e00d1790 con 0x7f47e015c460 2023-07-08T12:39:23.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47dd59a700 1 -- 172.21.15.12:0/3564303726 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c80035b0 con 0x7f47e015c460 2023-07-08T12:39:23.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47dd59a700 1 -- 172.21.15.12:0/3564303726 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582299395 0 0) 0x7f47e00cd950 con 0x7f47e00cf8f0 2023-07-08T12:39:23.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47dd59a700 1 -- 172.21.15.12:0/3564303726 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e00d1790 con 0x7f47e00cf8f0 2023-07-08T12:39:23.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47ddd9b700 1 -- 172.21.15.12:0/3564303726 >> v1:172.21.15.160:6789/0 conn(0x7f47e00cf8f0 legacy=0x7f47e015fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:23.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47ddd9b700 1 -- 172.21.15.12:0/3564303726 >> v1:172.21.15.12:6789/0 conn(0x7f47e015c460 legacy=0x7f47e00cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:23.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.886+0000 7f47ddd9b700 1 -- 172.21.15.12:0/3564303726 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e00d14b0 con 0x7f47e00d2640 2023-07-08T12:39:23.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.887+0000 7f47ddd9b700 1 --2- 172.21.15.12:0/3564303726 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47e00d2640 0x7f47e00b59d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009870 tx=0x7f47d4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1dc7300989532dd server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:23.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.887+0000 7f47dd59a700 1 -- 172.21.15.12:0/3564303726 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4011830 con 0x7f47e00d2640 2023-07-08T12:39:23.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.887+0000 7f47dd59a700 1 -- 172.21.15.12:0/3564303726 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d4011990 con 0x7f47e00d2640 2023-07-08T12:39:23.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.887+0000 7f47dd59a700 1 -- 172.21.15.12:0/3564303726 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4011d00 con 0x7f47e00d2640 2023-07-08T12:39:23.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.887+0000 7f47e4fcd700 1 -- 172.21.15.12:0/3564303726 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47e00d2640 msgr2=0x7f47e00b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.887+0000 7f47e4fcd700 1 --2- 172.21.15.12:0/3564303726 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47e00d2640 0x7f47e00b59d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009870 tx=0x7f47d4009b80 comp rx=0 tx=0).stop 2023-07-08T12:39:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.887+0000 7f47e4fcd700 1 -- 172.21.15.12:0/3564303726 shutdown_connections 2023-07-08T12:39:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.887+0000 7f47e4fcd700 1 --2- 172.21.15.12:0/3564303726 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47e00d2640 0x7f47e00b59d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.887+0000 7f47e4fcd700 1 -- 172.21.15.12:0/3564303726 >> 172.21.15.12:0/3564303726 conn(0x7f47e0071660 msgr2=0x7f47e0071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.887+0000 7f47e4fcd700 1 -- 172.21.15.12:0/3564303726 shutdown_connections 2023-07-08T12:39:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.887+0000 7f47e4fcd700 1 -- 172.21.15.12:0/3564303726 wait complete. 2023-07-08T12:39:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.889+0000 7f47e4fcd700 1 Processor -- start 2023-07-08T12:39:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.889+0000 7f47e4fcd700 1 -- start start 2023-07-08T12:39:23.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.890+0000 7f47e4fcd700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47e00cf8f0 0x7f47e01f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:23.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.890+0000 7f47e4fcd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e00d2640 0x7f47e00cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:23.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.890+0000 7f47e4fcd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e015c460 0x7f47e015e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:23.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.890+0000 7f47e4fcd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0159410 con 0x7f47e00d2640 2023-07-08T12:39:23.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.890+0000 7f47e4fcd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e01f2550 con 0x7f47e015c460 2023-07-08T12:39:23.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.890+0000 7f47e4fcd700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e01f2830 con 0x7f47e00cf8f0 2023-07-08T12:39:23.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.890+0000 7f47ddd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e00d2640 0x7f47e00cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:23.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.890+0000 7f47ddd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e00d2640 0x7f47e00cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51490/0 (socket says 172.21.15.12:51490) 2023-07-08T12:39:23.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.890+0000 7f47ddd9b700 1 -- 172.21.15.12:0/2186936075 learned_addr learned my addr 172.21.15.12:0/2186936075 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:23.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.890+0000 7f47de59c700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47e00cf8f0 0x7f47e01f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:23.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.890+0000 7f47ded9d700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e015c460 0x7f47e015e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:23.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.891+0000 7f47ddd9b700 1 -- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47e00cf8f0 msgr2=0x7f47e01f1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:23.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.891+0000 7f47ddd9b700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47e00cf8f0 0x7f47e01f1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:23.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.891+0000 7f47ddd9b700 1 -- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e015c460 msgr2=0x7f47e015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:23.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.891+0000 7f47ddd9b700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e015c460 0x7f47e015e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:23.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.891+0000 7f47ddd9b700 1 -- 172.21.15.12:0/2186936075 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d40094a0 con 0x7f47e00d2640 2023-07-08T12:39:23.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.891+0000 7f47de59c700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47e00cf8f0 0x7f47e01f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:39:23.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.891+0000 7f47ddd9b700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e00d2640 0x7f47e00cc5b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f47d40090b0 tx=0x7f47d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:23.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.891+0000 7f47ceffd700 1 -- 172.21.15.12:0/2186936075 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4004550 con 0x7f47e00d2640 2023-07-08T12:39:23.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.891+0000 7f47ceffd700 1 -- 172.21.15.12:0/2186936075 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d401f070 con 0x7f47e00d2640 2023-07-08T12:39:23.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.891+0000 7f47e4fcd700 1 -- 172.21.15.12:0/2186936075 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e01f2b10 con 0x7f47e00d2640 2023-07-08T12:39:23.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.891+0000 7f47ceffd700 1 -- 172.21.15.12:0/2186936075 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d40038e0 con 0x7f47e00d2640 2023-07-08T12:39:23.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.891+0000 7f47e4fcd700 1 -- 172.21.15.12:0/2186936075 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e015f6f0 con 0x7f47e00d2640 2023-07-08T12:39:23.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.892+0000 7f47e4fcd700 1 -- 172.21.15.12:0/2186936075 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e0159580 con 0x7f47e00d2640 2023-07-08T12:39:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.892+0000 7f47ceffd700 1 -- 172.21.15.12:0/2186936075 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f47d4003a40 con 0x7f47e00d2640 2023-07-08T12:39:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.893+0000 7f47ceffd700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f47c006ace0 0x7f47c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.893+0000 7f47ceffd700 1 -- 172.21.15.12:0/2186936075 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f47d4021070 con 0x7f47e00d2640 2023-07-08T12:39:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.893+0000 7f47de59c700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f47c006ace0 0x7f47c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.894+0000 7f47de59c700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f47c006ace0 0x7f47c006d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f47d0007fd0 tx=0x7f47d0007e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:23.897+0000 7f47ceffd700 1 -- 172.21.15.12:0/2186936075 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f47e0159580 con 0x7f47e00d2640 2023-07-08T12:39:24.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.111+0000 7f47e4fcd700 1 -- 172.21.15.12:0/2186936075 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f47e0159550 con 0x7f47e00d2640 2023-07-08T12:39:24.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.667+0000 7f47ceffd700 1 -- 172.21.15.12:0/2186936075 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v36) v1 ==== 97+0+0 (secure 0 0 0) 0x7f47d4055d70 con 0x7f47e00d2640 2023-07-08T12:39:24.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.670+0000 7f47e4fcd700 1 -- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f47c006ace0 msgr2=0x7f47c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:24.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.670+0000 7f47e4fcd700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f47c006ace0 0x7f47c006d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f47d0007fd0 tx=0x7f47d0007e00 comp rx=0 tx=0).stop 2023-07-08T12:39:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.670+0000 7f47e4fcd700 1 -- 172.21.15.12:0/2186936075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e00d2640 msgr2=0x7f47e00cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.670+0000 7f47e4fcd700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e00d2640 0x7f47e00cc5b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f47d40090b0 tx=0x7f47d4004750 comp rx=0 tx=0).stop 2023-07-08T12:39:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.670+0000 7f47e4fcd700 1 -- 172.21.15.12:0/2186936075 shutdown_connections 2023-07-08T12:39:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.670+0000 7f47e4fcd700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f47c006ace0 0x7f47c006d190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.670+0000 7f47e4fcd700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47e015c460 0x7f47e015e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.670+0000 7f47e4fcd700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47e00cf8f0 0x7f47e01f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.670+0000 7f47e4fcd700 1 --2- 172.21.15.12:0/2186936075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e00d2640 0x7f47e00cc5b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:24.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.670+0000 7f47e4fcd700 1 -- 172.21.15.12:0/2186936075 >> 172.21.15.12:0/2186936075 conn(0x7f47e0071660 msgr2=0x7f47e00d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:24.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.670+0000 7f47e4fcd700 1 -- 172.21.15.12:0/2186936075 shutdown_connections 2023-07-08T12:39:24.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.670+0000 7f47e4fcd700 1 -- 172.21.15.12:0/2186936075 wait complete. 2023-07-08T12:39:24.674 INFO:teuthology.orchestra.run.smithi012.stderr: removed all blocklist entries 2023-07-08T12:39:24.689 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-08T12:39:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.819+0000 7f2bef5fb700 1 Processor -- start 2023-07-08T12:39:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.819+0000 7f2bef5fb700 1 -- start start 2023-07-08T12:39:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2bef5fb700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2be8159c30 0x7f2be80cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2bef5fb700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2be8160790 0x7f2be806ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2bef5fb700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be815c890 con 0x7f2be8069f30 2023-07-08T12:39:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2bef5fb700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be815c730 con 0x7f2be8160790 2023-07-08T12:39:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2bef5fb700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be80cf590 con 0x7f2be8159c30 2023-07-08T12:39:24.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2becb96700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2be8160790 0x7f2be806ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:24.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2bedb98700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2be8069f30 0x7f2be815cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38896/0 (socket says 172.21.15.12:38896) 2023-07-08T12:39:24.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2becb96700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2be8160790 0x7f2be806ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51496/0 (socket says 172.21.15.12:51496) 2023-07-08T12:39:24.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2bedb98700 1 -- 172.21.15.12:0/1491385668 learned_addr learned my addr 172.21.15.12:0/1491385668 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:24.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2bed397700 1 --2- 172.21.15.12:0/1491385668 >> v2:172.21.15.160:3300/0 conn(0x7f2be8159c30 0x7f2be80cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:24.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2becb96700 1 -- 172.21.15.12:0/1491385668 >> v2:172.21.15.160:3300/0 conn(0x7f2be8159c30 msgr2=0x7f2be80cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:24.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2becb96700 1 --2- 172.21.15.12:0/1491385668 >> v2:172.21.15.160:3300/0 conn(0x7f2be8159c30 0x7f2be80cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:24.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2becb96700 1 -- 172.21.15.12:0/1491385668 >> v1:172.21.15.12:6789/0 conn(0x7f2be8069f30 legacy=0x7f2be815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:24.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2becb96700 1 -- 172.21.15.12:0/1491385668 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be8160c90 con 0x7f2be8160790 2023-07-08T12:39:24.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.820+0000 7f2becb96700 1 --2- 172.21.15.12:0/1491385668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be8160790 0x7f2be806ef10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2be8070fe0 tx=0x7f2bd8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a52a273c5225d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:24.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.821+0000 7f2bdffff700 1 -- 172.21.15.12:0/1491385668 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8011830 con 0x7f2be8160790 2023-07-08T12:39:24.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.821+0000 7f2bdffff700 1 -- 172.21.15.12:0/1491385668 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd8011990 con 0x7f2be8160790 2023-07-08T12:39:24.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.821+0000 7f2bdffff700 1 -- 172.21.15.12:0/1491385668 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8011d20 con 0x7f2be8160790 2023-07-08T12:39:24.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.821+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1491385668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be8160790 msgr2=0x7f2be806ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:24.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.821+0000 7f2bef5fb700 1 --2- 172.21.15.12:0/1491385668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be8160790 0x7f2be806ef10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2be8070fe0 tx=0x7f2bd8009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.821+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1491385668 shutdown_connections 2023-07-08T12:39:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.821+0000 7f2bef5fb700 1 --2- 172.21.15.12:0/1491385668 >> v2:172.21.15.160:3300/0 conn(0x7f2be8159c30 0x7f2be80cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.821+0000 7f2bef5fb700 1 --2- 172.21.15.12:0/1491385668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be8160790 0x7f2be806ef10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.821+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1491385668 >> 172.21.15.12:0/1491385668 conn(0x7f2be80cc290 msgr2=0x7f2be80cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.821+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1491385668 shutdown_connections 2023-07-08T12:39:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.821+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1491385668 wait complete. 2023-07-08T12:39:24.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.823+0000 7f2bef5fb700 1 Processor -- start 2023-07-08T12:39:24.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.823+0000 7f2bef5fb700 1 -- start start 2023-07-08T12:39:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.823+0000 7f2bef5fb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2be8069f30 0x7f2be814da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.823+0000 7f2bef5fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be8159c30 0x7f2be80b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.823+0000 7f2bef5fb700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2be8160790 0x7f2be80b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.823+0000 7f2bef5fb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be814dfa0 con 0x7f2be8159c30 2023-07-08T12:39:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.823+0000 7f2bef5fb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80b37c0 con 0x7f2be8069f30 2023-07-08T12:39:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.823+0000 7f2bef5fb700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80b2bb0 con 0x7f2be8160790 2023-07-08T12:39:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.823+0000 7f2becb96700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be8159c30 0x7f2be80b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2becb96700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be8159c30 0x7f2be80b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51504/0 (socket says 172.21.15.12:51504) 2023-07-08T12:39:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2becb96700 1 -- 172.21.15.12:0/1305069512 learned_addr learned my addr 172.21.15.12:0/1305069512 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2bed397700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2be8069f30 0x7f2be814da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2bedb98700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2be8160790 0x7f2be80b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2becb96700 1 -- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2be8160790 msgr2=0x7f2be80b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2becb96700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2be8160790 0x7f2be80b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2becb96700 1 -- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2be8069f30 msgr2=0x7f2be814da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2becb96700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2be8069f30 0x7f2be814da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2becb96700 1 -- 172.21.15.12:0/1305069512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd80094a0 con 0x7f2be8159c30 2023-07-08T12:39:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2bed397700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2be8069f30 0x7f2be814da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2becb96700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be8159c30 0x7f2be80b1b20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80097c0 tx=0x7f2bd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2bddffb700 1 -- 172.21.15.12:0/1305069512 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd80038a0 con 0x7f2be8159c30 2023-07-08T12:39:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2bddffb700 1 -- 172.21.15.12:0/1305069512 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd801e070 con 0x7f2be8159c30 2023-07-08T12:39:24.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1305069512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be80b2e90 con 0x7f2be8159c30 2023-07-08T12:39:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2bddffb700 1 -- 172.21.15.12:0/1305069512 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8003c20 con 0x7f2be8159c30 2023-07-08T12:39:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.824+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1305069512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be80b3450 con 0x7f2be8159c30 2023-07-08T12:39:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.825+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1305069512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be8158c10 con 0x7f2be8159c30 2023-07-08T12:39:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.826+0000 7f2bddffb700 1 -- 172.21.15.12:0/1305069512 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2bd802d450 con 0x7f2be8159c30 2023-07-08T12:39:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.826+0000 7f2bddffb700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2bcc06ace0 0x7f2bcc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.826+0000 7f2bddffb700 1 -- 172.21.15.12:0/1305069512 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f2bd808a410 con 0x7f2be8159c30 2023-07-08T12:39:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.827+0000 7f2bed397700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2bcc06ace0 0x7f2bcc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.827+0000 7f2bed397700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2bcc06ace0 0x7f2bcc06d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2be40099f0 tx=0x7f2be4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:24.831+0000 7f2bddffb700 1 -- 172.21.15.12:0/1305069512 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2bd80556c0 con 0x7f2be8159c30 2023-07-08T12:39:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.115+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1305069512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2be80cd450 con 0x7f2be8159c30 2023-07-08T12:39:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.115+0000 7f2bddffb700 1 -- 172.21.15.12:0/1305069512 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f2bd8058ce0 con 0x7f2be8159c30 2023-07-08T12:39:25.118 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:25.118 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-07-08T12:39:25.118 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_dump": [ 2023-07-08T12:39:25.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.119 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "mds.a", 2023-07-08T12:39:25.119 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB0WKlk/OpHIRAAPuAogibhBDej8k8fnXpxpA==", 2023-07-08T12:39:25.119 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.119 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow", 2023-07-08T12:39:25.119 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow *", 2023-07-08T12:39:25.119 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow *", 2023-07-08T12:39:25.120 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.120 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.120 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "mds.b", 2023-07-08T12:39:25.120 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB0WKlkTUpCLhAA61h48ZyB7uUwQ/1Bgy91Pg==", 2023-07-08T12:39:25.121 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.121 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow", 2023-07-08T12:39:25.121 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow *", 2023-07-08T12:39:25.121 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow *", 2023-07-08T12:39:25.121 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.121 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.122 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "mds.c", 2023-07-08T12:39:25.122 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB0WKlk2Ej+KBAAGi85adawEIe5Bhyl2VRNyA==", 2023-07-08T12:39:25.122 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.122 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow", 2023-07-08T12:39:25.122 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow *", 2023-07-08T12:39:25.122 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow *", 2023-07-08T12:39:25.122 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.122 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.123 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "mds.d", 2023-07-08T12:39:25.123 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB0WKlkM3qlNRAApFyRs7mjHdiBE0yvy4piIA==", 2023-07-08T12:39:25.123 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.123 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow", 2023-07-08T12:39:25.123 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow *", 2023-07-08T12:39:25.124 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow *", 2023-07-08T12:39:25.124 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.124 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.124 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.0", 2023-07-08T12:39:25.124 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB4WKlkT0erEhAAT6jFqFu8qEqcCV4e1EUuiA==", 2023-07-08T12:39:25.125 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.125 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2023-07-08T12:39:25.125 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2023-07-08T12:39:25.125 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.125 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.126 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.1", 2023-07-08T12:39:25.126 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB7WKlkAiotFBAA4mjsombJttHweEpkU2+dNA==", 2023-07-08T12:39:25.126 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.126 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2023-07-08T12:39:25.126 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2023-07-08T12:39:25.126 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.126 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.127 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.2", 2023-07-08T12:39:25.127 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB+WKlk5OEWFRAAIJIWpT7tetRJCNe1COBOVA==", 2023-07-08T12:39:25.127 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.127 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2023-07-08T12:39:25.128 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2023-07-08T12:39:25.128 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.128 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.128 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.3", 2023-07-08T12:39:25.129 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQCBWKlkHcY8FhAAdHT4AbuyWeBD6dYkIoEBOQ==", 2023-07-08T12:39:25.129 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.129 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2023-07-08T12:39:25.129 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2023-07-08T12:39:25.129 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.129 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.130 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.4", 2023-07-08T12:39:25.130 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQCHWKlkJo3IHhAAl8N/CzlnspcQapOVcRjpIQ==", 2023-07-08T12:39:25.130 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.130 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2023-07-08T12:39:25.130 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2023-07-08T12:39:25.130 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.130 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.131 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.5", 2023-07-08T12:39:25.131 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQCKWKlkC9ImHhAAV4i9xjI3iPq4VSx01my0AQ==", 2023-07-08T12:39:25.131 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.131 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2023-07-08T12:39:25.131 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2023-07-08T12:39:25.131 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.132 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.132 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.6", 2023-07-08T12:39:25.132 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQCNWKlko4ijHRAAIQbuPDCHjpC4KbQayUdinQ==", 2023-07-08T12:39:25.132 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.133 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2023-07-08T12:39:25.133 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2023-07-08T12:39:25.133 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.133 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.133 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.7", 2023-07-08T12:39:25.134 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQCQWKlkOPO0HBAAJDU9sOOoFBqX1o8BFizu3w==", 2023-07-08T12:39:25.134 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.134 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2023-07-08T12:39:25.134 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2023-07-08T12:39:25.134 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.134 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.135 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.0", 2023-07-08T12:39:25.135 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB0WKlkvys7OhAA7XADn3L9fMd0Gow4lK3U0g==", 2023-07-08T12:39:25.135 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.135 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow", 2023-07-08T12:39:25.135 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow r", 2023-07-08T12:39:25.135 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow rw", 2023-07-08T12:39:25.135 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow rwx" 2023-07-08T12:39:25.136 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.136 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.1", 2023-07-08T12:39:25.136 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB1WKlkeYeZBRAAG9I1KVYLBQHMvW/zl1UBqQ==", 2023-07-08T12:39:25.136 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.137 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow", 2023-07-08T12:39:25.137 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow r", 2023-07-08T12:39:25.137 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow rw", 2023-07-08T12:39:25.137 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow rwx" 2023-07-08T12:39:25.137 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.138 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.2", 2023-07-08T12:39:25.138 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB1WKlkSEU1AhAAekux8ymGBJEH4ZjtKKD9mg==", 2023-07-08T12:39:25.138 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.138 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow", 2023-07-08T12:39:25.138 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow r", 2023-07-08T12:39:25.138 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow rw", 2023-07-08T12:39:25.138 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow rwx" 2023-07-08T12:39:25.138 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.139 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.3", 2023-07-08T12:39:25.139 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB1WKlkdc1BCxAAvYEEUhu/qoH4lZ31bg+D3A==", 2023-07-08T12:39:25.139 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.139 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow", 2023-07-08T12:39:25.140 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow r", 2023-07-08T12:39:25.140 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow rw", 2023-07-08T12:39:25.140 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow rwx" 2023-07-08T12:39:25.140 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.140 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.admin", 2023-07-08T12:39:25.141 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g==", 2023-07-08T12:39:25.141 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.141 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow *", 2023-07-08T12:39:25.141 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow *", 2023-07-08T12:39:25.141 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow *", 2023-07-08T12:39:25.141 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.141 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.142 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.bootstrap-mds", 2023-07-08T12:39:25.142 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQCWWKlkXV/rHRAAD9gpNvcR3mqTpD9KoXTlOQ==", 2023-07-08T12:39:25.142 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.142 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile bootstrap-mds" 2023-07-08T12:39:25.142 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.143 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.bootstrap-mgr", 2023-07-08T12:39:25.143 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQCWWKlkdGvrHRAAMGGisH90JaYdGOsm/W6vaA==", 2023-07-08T12:39:25.143 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.143 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile bootstrap-mgr" 2023-07-08T12:39:25.143 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.144 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.bootstrap-osd", 2023-07-08T12:39:25.144 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQCWWKlk+HTrHRAAThvI8nX/LlGJaF9kRzKw3Q==", 2023-07-08T12:39:25.144 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.144 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile bootstrap-osd" 2023-07-08T12:39:25.144 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.145 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.bootstrap-rbd", 2023-07-08T12:39:25.146 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQCWWKlkfX3rHRAASck5UKCE6lbppE12lLc+cw==", 2023-07-08T12:39:25.146 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.146 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile bootstrap-rbd" 2023-07-08T12:39:25.146 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.147 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-07-08T12:39:25.147 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQCWWKlk4IXrHRAAVwvonl2EOWA9D/ww5XcJvg==", 2023-07-08T12:39:25.148 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.148 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-07-08T12:39:25.149 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.149 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.bootstrap-rgw", 2023-07-08T12:39:25.150 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQCWWKlkYI7rHRAAzz/thy03Shv1ORI0imaBuQ==", 2023-07-08T12:39:25.150 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.150 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile bootstrap-rgw" 2023-07-08T12:39:25.151 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.151 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "mgr.x", 2023-07-08T12:39:25.152 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB0WKlkHzeqHRAATq0FhzPUkz0sKSEOl27AxQ==", 2023-07-08T12:39:25.152 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.152 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow *", 2023-07-08T12:39:25.152 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile mgr", 2023-07-08T12:39:25.152 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.153 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:25.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:25.153 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "mgr.y", 2023-07-08T12:39:25.154 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQB0WKlkaaigGRAAmINnVITHyAQihLVO3fEr2Q==", 2023-07-08T12:39:25.154 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2023-07-08T12:39:25.154 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow *", 2023-07-08T12:39:25.155 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile mgr", 2023-07-08T12:39:25.155 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2023-07-08T12:39:25.155 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.155 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:25.155 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-07-08T12:39:25.155 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-07-08T12:39:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.117+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2bcc06ace0 msgr2=0x7f2bcc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.117+0000 7f2bef5fb700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2bcc06ace0 0x7f2bcc06d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2be40099f0 tx=0x7f2be4008040 comp rx=0 tx=0).stop 2023-07-08T12:39:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.117+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1305069512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be8159c30 msgr2=0x7f2be80b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.117+0000 7f2bef5fb700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be8159c30 0x7f2be80b1b20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80097c0 tx=0x7f2bd8004750 comp rx=0 tx=0).stop 2023-07-08T12:39:25.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.118+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1305069512 shutdown_connections 2023-07-08T12:39:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.118+0000 7f2bef5fb700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2bcc06ace0 0x7f2bcc06d190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.118+0000 7f2bef5fb700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2be8160790 0x7f2be80b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.118+0000 7f2bef5fb700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2be8069f30 0x7f2be814da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.118+0000 7f2bef5fb700 1 --2- 172.21.15.12:0/1305069512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be8159c30 0x7f2be80b1b20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.118+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1305069512 >> 172.21.15.12:0/1305069512 conn(0x7f2be80cc290 msgr2=0x7f2be8158f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.118+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1305069512 shutdown_connections 2023-07-08T12:39:25.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.118+0000 7f2bef5fb700 1 -- 172.21.15.12:0/1305069512 wait complete. 2023-07-08T12:39:25.160 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-07-08T12:39:25.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.284+0000 7fef0694a700 1 Processor -- start 2023-07-08T12:39:25.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.284+0000 7fef0694a700 1 -- start start 2023-07-08T12:39:25.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7fef0694a700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fef00159130 0x7fef000adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7fef0694a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fef000ae730 0x7fef000cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7fef0694a700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef000af750 con 0x7fef000cd960 2023-07-08T12:39:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7fef0694a700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef000ae5a0 con 0x7fef000ae730 2023-07-08T12:39:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7fef0694a700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef000d0760 con 0x7fef00159130 2023-07-08T12:39:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7feeff7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fef000ae730 0x7fef000cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7feeff7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fef000ae730 0x7fef000cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51510/0 (socket says 172.21.15.12:51510) 2023-07-08T12:39:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7feeff7fe700 1 -- 172.21.15.12:0/1368445860 learned_addr learned my addr 172.21.15.12:0/1368445860 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7feeff7fe700 1 -- 172.21.15.12:0/1368445860 >> v2:172.21.15.160:3300/0 conn(0x7fef00159130 msgr2=0x7fef000adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7feefffff700 1 --2- 172.21.15.12:0/1368445860 >> v2:172.21.15.160:3300/0 conn(0x7fef00159130 0x7fef000adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7feeff7fe700 1 --2- 172.21.15.12:0/1368445860 >> v2:172.21.15.160:3300/0 conn(0x7fef00159130 0x7fef000adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7feeff7fe700 1 -- 172.21.15.12:0/1368445860 >> v1:172.21.15.160:6789/0 conn(0x7fef000cd960 legacy=0x7fef000ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7feeff7fe700 1 -- 172.21.15.12:0/1368445860 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef000cdd80 con 0x7fef000ae730 2023-07-08T12:39:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.285+0000 7feeff7fe700 1 --2- 172.21.15.12:0/1368445860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef000ae730 0x7fef000cd420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fef00070c80 tx=0x7feee8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45b854ef7b14c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.286+0000 7feefeffd700 1 -- 172.21.15.12:0/1368445860 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8004230 con 0x7fef000ae730 2023-07-08T12:39:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.286+0000 7feefeffd700 1 -- 172.21.15.12:0/1368445860 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee8004390 con 0x7fef000ae730 2023-07-08T12:39:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.286+0000 7feefeffd700 1 -- 172.21.15.12:0/1368445860 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee80036a0 con 0x7fef000ae730 2023-07-08T12:39:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.286+0000 7fef0694a700 1 -- 172.21.15.12:0/1368445860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef000ae730 msgr2=0x7fef000cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.286+0000 7fef0694a700 1 --2- 172.21.15.12:0/1368445860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef000ae730 0x7fef000cd420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fef00070c80 tx=0x7feee8009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.286+0000 7fef0694a700 1 -- 172.21.15.12:0/1368445860 shutdown_connections 2023-07-08T12:39:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.286+0000 7fef0694a700 1 --2- 172.21.15.12:0/1368445860 >> v2:172.21.15.160:3300/0 conn(0x7fef00159130 0x7fef000adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.286+0000 7fef0694a700 1 --2- 172.21.15.12:0/1368445860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef000ae730 0x7fef000cd420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.286+0000 7fef0694a700 1 -- 172.21.15.12:0/1368445860 >> 172.21.15.12:0/1368445860 conn(0x7fef00098a30 msgr2=0x7fef000cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.286+0000 7fef0694a700 1 -- 172.21.15.12:0/1368445860 shutdown_connections 2023-07-08T12:39:25.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.286+0000 7fef0694a700 1 -- 172.21.15.12:0/1368445860 wait complete. 2023-07-08T12:39:25.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.287+0000 7fef0694a700 1 Processor -- start 2023-07-08T12:39:25.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.288+0000 7fef0694a700 1 -- start start 2023-07-08T12:39:25.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.288+0000 7fef0694a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef000ae730 0x7fef000b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.288+0000 7fef0694a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef000cd960 0x7fef001f09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.288+0000 7fef0694a700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fef00159130 0x7fef00154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.288+0000 7fef0694a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef001552d0 con 0x7fef000ae730 2023-07-08T12:39:25.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.288+0000 7fef0694a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef001f35e0 con 0x7fef000cd960 2023-07-08T12:39:25.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.288+0000 7fef0694a700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef001f38c0 con 0x7fef00159130 2023-07-08T12:39:25.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.288+0000 7feefffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef000ae730 0x7fef000b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.288+0000 7feefffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef000ae730 0x7fef000b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51514/0 (socket says 172.21.15.12:51514) 2023-07-08T12:39:25.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.288+0000 7feefffff700 1 -- 172.21.15.12:0/675996961 learned_addr learned my addr 172.21.15.12:0/675996961 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:25.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.289+0000 7fef04ee7700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fef00159130 0x7fef00154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.289+0000 7feeff7fe700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef000cd960 0x7fef001f09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.289+0000 7feefffff700 1 -- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fef00159130 msgr2=0x7fef00154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:25.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.289+0000 7feefffff700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fef00159130 0x7fef00154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.289+0000 7feefffff700 1 -- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef000cd960 msgr2=0x7fef001f09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:25.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.289+0000 7feefffff700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef000cd960 0x7fef001f09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.289+0000 7feefffff700 1 -- 172.21.15.12:0/675996961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee8009550 con 0x7fef000ae730 2023-07-08T12:39:25.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.289+0000 7feefffff700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef000ae730 0x7fef000b47d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7feef000c7f0 tx=0x7feef000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:25.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.289+0000 7feefcff9700 1 -- 172.21.15.12:0/675996961 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0011800 con 0x7fef000ae730 2023-07-08T12:39:25.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.289+0000 7feefcff9700 1 -- 172.21.15.12:0/675996961 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef0011960 con 0x7fef000ae730 2023-07-08T12:39:25.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.289+0000 7fef0694a700 1 -- 172.21.15.12:0/675996961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef001f3ba0 con 0x7fef000ae730 2023-07-08T12:39:25.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.289+0000 7feefcff9700 1 -- 172.21.15.12:0/675996961 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0011cf0 con 0x7fef000ae730 2023-07-08T12:39:25.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.289+0000 7fef0694a700 1 -- 172.21.15.12:0/675996961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef001f4160 con 0x7fef000ae730 2023-07-08T12:39:25.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.290+0000 7fef0694a700 1 -- 172.21.15.12:0/675996961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef000c8c90 con 0x7fef000ae730 2023-07-08T12:39:25.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.291+0000 7feefcff9700 1 -- 172.21.15.12:0/675996961 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feef0018360 con 0x7fef000ae730 2023-07-08T12:39:25.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.291+0000 7feefcff9700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7feee006ace0 0x7feee006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.291+0000 7feefcff9700 1 -- 172.21.15.12:0/675996961 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7feef00892a0 con 0x7fef000ae730 2023-07-08T12:39:25.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.291+0000 7feeff7fe700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7feee006ace0 0x7feee006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.292+0000 7feeff7fe700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7feee006ace0 0x7feee006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7feee8009130 tx=0x7feee8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:25.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.295+0000 7feefcff9700 1 -- 172.21.15.12:0/675996961 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feef0058000 con 0x7fef000ae730 2023-07-08T12:39:25.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.576+0000 7fef0694a700 1 -- 172.21.15.12:0/675996961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fef0004ea60 con 0x7fef000ae730 2023-07-08T12:39:25.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.580+0000 7feefcff9700 1 -- 172.21.15.12:0/675996961 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7feef0057b90 con 0x7fef000ae730 2023-07-08T12:39:25.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.582+0000 7fef0694a700 1 -- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7feee006ace0 msgr2=0x7feee006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:25.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.582+0000 7fef0694a700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7feee006ace0 0x7feee006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7feee8009130 tx=0x7feee8008030 comp rx=0 tx=0).stop 2023-07-08T12:39:25.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.582+0000 7fef0694a700 1 -- 172.21.15.12:0/675996961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef000ae730 msgr2=0x7fef000b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:25.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.582+0000 7fef0694a700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef000ae730 0x7fef000b47d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7feef000c7f0 tx=0x7feef000cbb0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.582+0000 7fef0694a700 1 -- 172.21.15.12:0/675996961 shutdown_connections 2023-07-08T12:39:25.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.582+0000 7fef0694a700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7feee006ace0 0x7feee006d190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.582+0000 7fef0694a700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fef00159130 0x7fef00154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.582+0000 7fef0694a700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef000ae730 0x7fef000b47d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.582+0000 7fef0694a700 1 --2- 172.21.15.12:0/675996961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef000cd960 0x7fef001f09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.582+0000 7fef0694a700 1 -- 172.21.15.12:0/675996961 >> 172.21.15.12:0/675996961 conn(0x7fef00098a30 msgr2=0x7fef0006e090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:25.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.582+0000 7fef0694a700 1 -- 172.21.15.12:0/675996961 shutdown_connections 2023-07-08T12:39:25.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.582+0000 7fef0694a700 1 -- 172.21.15.12:0/675996961 wait complete. 2023-07-08T12:39:25.597 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-07-08T12:39:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.724+0000 7f4034c44700 1 Processor -- start 2023-07-08T12:39:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.724+0000 7f4034c44700 1 -- start start 2023-07-08T12:39:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.725+0000 7f4034c44700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f403015a3a0 0x7f403015e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.725+0000 7f4034c44700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403015a770 con 0x7f40300ca2b0 2023-07-08T12:39:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.725+0000 7f4034c44700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403015e500 con 0x7f40300c9e60 2023-07-08T12:39:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.725+0000 7f4034c44700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40300cbd10 con 0x7f403015a3a0 2023-07-08T12:39:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.725+0000 7f402ed9d700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f40300ca2b0 0x7f403006ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38922/0 (socket says 172.21.15.12:38922) 2023-07-08T12:39:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402ed9d700 1 -- 172.21.15.12:0/3768863995 learned_addr learned my addr 172.21.15.12:0/3768863995 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402e59c700 1 --2- 172.21.15.12:0/3768863995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f403015a3a0 0x7f403015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779569673 0 0) 0x7f4028001490 con 0x7f40300ca2b0 2023-07-08T12:39:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f401c0035b0 con 0x7f40300ca2b0 2023-07-08T12:39:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2735706993 0 0) 0x7f401c0035b0 con 0x7f40300ca2b0 2023-07-08T12:39:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f403015a3a0 msgr2=0x7f403015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 --2- 172.21.15.12:0/3768863995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f403015a3a0 0x7f403015e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 >> v1:172.21.15.160:6789/0 conn(0x7f40300c9e60 legacy=0x7f4030069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403015deb0 con 0x7f40300ca2b0 2023-07-08T12:39:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f4028001b80 con 0x7f40300ca2b0 2023-07-08T12:39:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 >> v1:172.21.15.12:6789/0 conn(0x7f40300ca2b0 legacy=0x7f403006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 --2- 172.21.15.12:0/3768863995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f401c003a30 0x7f401c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 --2- 172.21.15.12:0/3768863995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f403015a3a0 0x7f401c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 --2- 172.21.15.12:0/3768863995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f401c004d90 0x7f401c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300cbd10 con 0x7f401c004d90 2023-07-08T12:39:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c00bac0 con 0x7f401c003a30 2023-07-08T12:39:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c00bc50 con 0x7f403015a3a0 2023-07-08T12:39:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402ed9d700 1 --2- 172.21.15.12:0/3768863995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f401c004d90 0x7f401c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402e59c700 1 --2- 172.21.15.12:0/3768863995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f401c003a30 0x7f401c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.726+0000 7f402dd9b700 1 --2- 172.21.15.12:0/3768863995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f403015a3a0 0x7f401c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.727+0000 7f402ed9d700 1 -- 172.21.15.12:0/3768863995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f403015a3a0 msgr2=0x7f401c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.727+0000 7f402ed9d700 1 --2- 172.21.15.12:0/3768863995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f403015a3a0 0x7f401c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.727+0000 7f402ed9d700 1 -- 172.21.15.12:0/3768863995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f401c003a30 msgr2=0x7f401c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:25.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.727+0000 7f402ed9d700 1 --2- 172.21.15.12:0/3768863995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f401c003a30 0x7f401c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.727+0000 7f402ed9d700 1 -- 172.21.15.12:0/3768863995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0035b0 con 0x7f401c004d90 2023-07-08T12:39:25.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.727+0000 7f402ed9d700 1 --2- 172.21.15.12:0/3768863995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f401c004d90 0x7f401c005200 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f402800a830 tx=0x7f401c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f00f253524924de server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:25.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.727+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402800a490 con 0x7f401c004d90 2023-07-08T12:39:25.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.727+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4028012620 con 0x7f401c004d90 2023-07-08T12:39:25.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.727+0000 7f402d59a700 1 -- 172.21.15.12:0/3768863995 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280129d0 con 0x7f401c004d90 2023-07-08T12:39:25.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.727+0000 7f4034c44700 1 -- 172.21.15.12:0/3768863995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f401c004d90 msgr2=0x7f401c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:25.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.727+0000 7f4034c44700 1 --2- 172.21.15.12:0/3768863995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f401c004d90 0x7f401c005200 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f402800a830 tx=0x7f401c005880 comp rx=0 tx=0).stop 2023-07-08T12:39:25.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.728+0000 7f402ed9d700 1 -- 172.21.15.12:0/3768863995 reap_dead start 2023-07-08T12:39:25.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.728+0000 7f4034c44700 1 -- 172.21.15.12:0/3768863995 shutdown_connections 2023-07-08T12:39:25.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.728+0000 7f4034c44700 1 -- 172.21.15.12:0/3768863995 >> 172.21.15.12:0/3768863995 conn(0x7f40300cc290 msgr2=0x7f40300cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:25.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.728+0000 7f4034c44700 1 -- 172.21.15.12:0/3768863995 shutdown_connections 2023-07-08T12:39:25.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.728+0000 7f4034c44700 1 -- 172.21.15.12:0/3768863995 wait complete. 2023-07-08T12:39:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.729+0000 7f4034c44700 1 Processor -- start 2023-07-08T12:39:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.729+0000 7f4034c44700 1 -- start start 2023-07-08T12:39:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f4034c44700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f401c004d90 0x7f40300c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f4034c44700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40301f3910 0x7f40301f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f4034c44700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40300c4840 0x7f40300c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f4034c44700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300c5b30 con 0x7f401c004d90 2023-07-08T12:39:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f4034c44700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40301f42c0 con 0x7f40301f3910 2023-07-08T12:39:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f4034c44700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403015ae20 con 0x7f40300c4840 2023-07-08T12:39:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f402e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f401c004d90 0x7f40300c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f402e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f401c004d90 0x7f40300c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51540/0 (socket says 172.21.15.12:51540) 2023-07-08T12:39:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f402e59c700 1 -- 172.21.15.12:0/2017725299 learned_addr learned my addr 172.21.15.12:0/2017725299 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f402ed9d700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40300c4840 0x7f40300c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f402dd9b700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40301f3910 0x7f40301f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f402e59c700 1 -- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40300c4840 msgr2=0x7f40300c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f402e59c700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40300c4840 0x7f40300c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f402e59c700 1 -- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40301f3910 msgr2=0x7f40301f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f402e59c700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40301f3910 0x7f40301f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f402e59c700 1 -- 172.21.15.12:0/2017725299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40300c5570 con 0x7f401c004d90 2023-07-08T12:39:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f402dd9b700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40301f3910 0x7f40301f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:25.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.730+0000 7f402e59c700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f401c004d90 0x7f40300c90b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4020011ad0 tx=0x7f4020011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:25.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.731+0000 7f4026ffd700 1 -- 172.21.15.12:0/2017725299 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402000fa00 con 0x7f401c004d90 2023-07-08T12:39:25.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.731+0000 7f4026ffd700 1 -- 172.21.15.12:0/2017725299 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402000fb60 con 0x7f401c004d90 2023-07-08T12:39:25.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.731+0000 7f4034c44700 1 -- 172.21.15.12:0/2017725299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40300c5850 con 0x7f401c004d90 2023-07-08T12:39:25.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.731+0000 7f4026ffd700 1 -- 172.21.15.12:0/2017725299 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020014440 con 0x7f401c004d90 2023-07-08T12:39:25.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.731+0000 7f4034c44700 1 -- 172.21.15.12:0/2017725299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40300c95f0 con 0x7f401c004d90 2023-07-08T12:39:25.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.732+0000 7f4034c44700 1 -- 172.21.15.12:0/2017725299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403015a910 con 0x7f401c004d90 2023-07-08T12:39:25.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.732+0000 7f4026ffd700 1 -- 172.21.15.12:0/2017725299 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f40200145a0 con 0x7f401c004d90 2023-07-08T12:39:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.732+0000 7f4026ffd700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f401006ace0 0x7f401006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.733+0000 7f4026ffd700 1 -- 172.21.15.12:0/2017725299 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f4020089b20 con 0x7f401c004d90 2023-07-08T12:39:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.733+0000 7f402dd9b700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f401006ace0 0x7f401006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.733+0000 7f402dd9b700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f401006ace0 0x7f401006d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4030153eb0 tx=0x7f4018009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:25.737+0000 7f4026ffd700 1 -- 172.21.15.12:0/2017725299 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f403015a910 con 0x7f401c004d90 2023-07-08T12:39:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.015+0000 7f4034c44700 1 -- 172.21.15.12:0/2017725299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f403015a910 con 0x7f401c004d90 2023-07-08T12:39:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.019+0000 7f4026ffd700 1 -- 172.21.15.12:0/2017725299 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f403015a910 con 0x7f401c004d90 2023-07-08T12:39:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.021+0000 7f4034c44700 1 -- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f401006ace0 msgr2=0x7f401006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.021+0000 7f4034c44700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f401006ace0 0x7f401006d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4030153eb0 tx=0x7f4018009040 comp rx=0 tx=0).stop 2023-07-08T12:39:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.021+0000 7f4034c44700 1 -- 172.21.15.12:0/2017725299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f401c004d90 msgr2=0x7f40300c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.021+0000 7f4034c44700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f401c004d90 0x7f40300c90b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4020011ad0 tx=0x7f4020011e90 comp rx=0 tx=0).stop 2023-07-08T12:39:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.021+0000 7f4034c44700 1 -- 172.21.15.12:0/2017725299 shutdown_connections 2023-07-08T12:39:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.021+0000 7f4034c44700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f401006ace0 0x7f401006d190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.021+0000 7f4034c44700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40300c4840 0x7f40300c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.021+0000 7f4034c44700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f401c004d90 0x7f40300c90b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.021+0000 7f4034c44700 1 --2- 172.21.15.12:0/2017725299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40301f3910 0x7f40301f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.021+0000 7f4034c44700 1 -- 172.21.15.12:0/2017725299 >> 172.21.15.12:0/2017725299 conn(0x7f40300cc290 msgr2=0x7f40300b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.021+0000 7f4034c44700 1 -- 172.21.15.12:0/2017725299 shutdown_connections 2023-07-08T12:39:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.021+0000 7f4034c44700 1 -- 172.21.15.12:0/2017725299 wait complete. 2023-07-08T12:39:26.036 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-07-08T12:39:26.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.160+0000 7f6746fb7700 1 Processor -- start 2023-07-08T12:39:26.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.160+0000 7f6746fb7700 1 -- start start 2023-07-08T12:39:26.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.160+0000 7f6746fb7700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f67400ce1b0 0x7f67400cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:26.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.160+0000 7f6746fb7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f674015d6c0 0x7f6740071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:26.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.160+0000 7f6746fb7700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674015d230 con 0x7f67400b1c70 2023-07-08T12:39:26.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.160+0000 7f6746fb7700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67400c9e60 con 0x7f67400ce1b0 2023-07-08T12:39:26.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.160+0000 7f6746fb7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67400cd920 con 0x7f674015d6c0 2023-07-08T12:39:26.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.161+0000 7f6744d53700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f67400ce1b0 0x7f67400cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:26.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.161+0000 7f6745554700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f674015d6c0 0x7f6740071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:26.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.161+0000 7f673ffff700 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f67400b1c70 0x7f67400ce580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.12:48164/0 (socket says 172.21.15.12:48164) 2023-07-08T12:39:26.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.161+0000 7f6745554700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f674015d6c0 0x7f6740071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.12:32936/0 (socket says 172.21.15.12:32936) 2023-07-08T12:39:26.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.161+0000 7f673ffff700 1 -- 172.21.15.12:0/3669857480 learned_addr learned my addr 172.21.15.12:0/3669857480 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:26.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.161+0000 7f6744d53700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f67400ce1b0 0x7f67400cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50290/0 (socket says 172.21.15.12:50290) 2023-07-08T12:39:26.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.161+0000 7f6745554700 1 -- 172.21.15.12:0/3669857480 >> v2:172.21.15.160:3300/0 conn(0x7f67400ce1b0 msgr2=0x7f67400cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:26.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.161+0000 7f6745554700 1 --2- 172.21.15.12:0/3669857480 >> v2:172.21.15.160:3300/0 conn(0x7f67400ce1b0 0x7f67400cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.161+0000 7f6745554700 1 -- 172.21.15.12:0/3669857480 >> v1:172.21.15.160:6789/0 conn(0x7f67400b1c70 legacy=0x7f67400ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.161+0000 7f6745554700 1 -- 172.21.15.12:0/3669857480 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67400cdbc0 con 0x7f674015d6c0 2023-07-08T12:39:26.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.161+0000 7f6744d53700 1 --2- 172.21.15.12:0/3669857480 >> v2:172.21.15.160:3300/0 conn(0x7f67400ce1b0 0x7f67400cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:39:26.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.161+0000 7f6745554700 1 --2- 172.21.15.12:0/3669857480 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f674015d6c0 0x7f6740071530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6738009d90 tx=0x7f673800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45e1227967b93040 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:26.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.161+0000 7f673f7fe700 1 -- 172.21.15.12:0/3669857480 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673800e070 con 0x7f674015d6c0 2023-07-08T12:39:26.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.162+0000 7f673f7fe700 1 -- 172.21.15.12:0/3669857480 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673800bb20 con 0x7f674015d6c0 2023-07-08T12:39:26.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.162+0000 7f673f7fe700 1 -- 172.21.15.12:0/3669857480 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738012430 con 0x7f674015d6c0 2023-07-08T12:39:26.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.162+0000 7f6746fb7700 1 -- 172.21.15.12:0/3669857480 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f674015d6c0 msgr2=0x7f6740071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.162+0000 7f6746fb7700 1 --2- 172.21.15.12:0/3669857480 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f674015d6c0 0x7f6740071530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6738009d90 tx=0x7f673800b330 comp rx=0 tx=0).stop 2023-07-08T12:39:26.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.162+0000 7f6746fb7700 1 -- 172.21.15.12:0/3669857480 shutdown_connections 2023-07-08T12:39:26.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.162+0000 7f6746fb7700 1 --2- 172.21.15.12:0/3669857480 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f674015d6c0 0x7f6740071530 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.162+0000 7f6746fb7700 1 --2- 172.21.15.12:0/3669857480 >> v2:172.21.15.160:3300/0 conn(0x7f67400ce1b0 0x7f67400cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.162+0000 7f6746fb7700 1 -- 172.21.15.12:0/3669857480 >> 172.21.15.12:0/3669857480 conn(0x7f67400cc290 msgr2=0x7f67400cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:26.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.162+0000 7f6746fb7700 1 -- 172.21.15.12:0/3669857480 shutdown_connections 2023-07-08T12:39:26.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.162+0000 7f6746fb7700 1 -- 172.21.15.12:0/3669857480 wait complete. 2023-07-08T12:39:26.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.163+0000 7f6746fb7700 1 Processor -- start 2023-07-08T12:39:26.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.163+0000 7f6746fb7700 1 -- start start 2023-07-08T12:39:26.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.164+0000 7f6746fb7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67400b1c70 0x7f6740157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:26.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.164+0000 7f6746fb7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67400ce1b0 0x7f67400ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:26.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.164+0000 7f6746fb7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f674015d6c0 0x7f67400abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:26.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.164+0000 7f6746fb7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67400aece0 con 0x7f674015d6c0 2023-07-08T12:39:26.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.164+0000 7f6746fb7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67400c5880 con 0x7f67400ce1b0 2023-07-08T12:39:26.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.164+0000 7f6746fb7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67400c4d00 con 0x7f67400b1c70 2023-07-08T12:39:26.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f6745554700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f674015d6c0 0x7f67400abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:26.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f6745554700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f674015d6c0 0x7f67400abdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51556/0 (socket says 172.21.15.12:51556) 2023-07-08T12:39:26.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f6745554700 1 -- 172.21.15.12:0/1208671275 learned_addr learned my addr 172.21.15.12:0/1208671275 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:26.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f6744d53700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67400b1c70 0x7f6740157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:26.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f673ffff700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67400ce1b0 0x7f67400ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:26.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f6745554700 1 -- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67400b1c70 msgr2=0x7f6740157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f6745554700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67400b1c70 0x7f6740157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f6745554700 1 -- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67400ce1b0 msgr2=0x7f67400ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f6745554700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67400ce1b0 0x7f67400ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f6745554700 1 -- 172.21.15.12:0/1208671275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67340094a0 con 0x7f674015d6c0 2023-07-08T12:39:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f6745554700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f674015d6c0 0x7f67400abdf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6738014040 tx=0x7f6738004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f673d7fa700 1 -- 172.21.15.12:0/1208671275 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738012430 con 0x7f674015d6c0 2023-07-08T12:39:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f673d7fa700 1 -- 172.21.15.12:0/1208671275 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6738020070 con 0x7f674015d6c0 2023-07-08T12:39:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f6746fb7700 1 -- 172.21.15.12:0/1208671275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6738009a70 con 0x7f674015d6c0 2023-07-08T12:39:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f673d7fa700 1 -- 172.21.15.12:0/1208671275 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67380098f0 con 0x7f674015d6c0 2023-07-08T12:39:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.165+0000 7f6746fb7700 1 -- 172.21.15.12:0/1208671275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67400c52c0 con 0x7f674015d6c0 2023-07-08T12:39:26.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.166+0000 7f6746fb7700 1 -- 172.21.15.12:0/1208671275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674004ea60 con 0x7f674015d6c0 2023-07-08T12:39:26.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.167+0000 7f673d7fa700 1 -- 172.21.15.12:0/1208671275 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6738004040 con 0x7f674015d6c0 2023-07-08T12:39:26.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.167+0000 7f673d7fa700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f672406ace0 0x7f672406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:26.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.167+0000 7f673d7fa700 1 -- 172.21.15.12:0/1208671275 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6738022070 con 0x7f674015d6c0 2023-07-08T12:39:26.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.167+0000 7f6744d53700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f672406ace0 0x7f672406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:26.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.168+0000 7f6744d53700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f672406ace0 0x7f672406d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f67400b1420 tx=0x7f6734009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:26.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.171+0000 7f673d7fa700 1 -- 172.21.15.12:0/1208671275 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f67380561b0 con 0x7f674015d6c0 2023-07-08T12:39:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.455+0000 7f6746fb7700 1 -- 172.21.15.12:0/1208671275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f67400b5790 con 0x7f674015d6c0 2023-07-08T12:39:26.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.461+0000 7f673d7fa700 1 -- 172.21.15.12:0/1208671275 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f67380597d0 con 0x7f674015d6c0 2023-07-08T12:39:26.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.463+0000 7f6746fb7700 1 -- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f672406ace0 msgr2=0x7f672406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.463+0000 7f6746fb7700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f672406ace0 0x7f672406d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f67400b1420 tx=0x7f6734009210 comp rx=0 tx=0).stop 2023-07-08T12:39:26.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.463+0000 7f6746fb7700 1 -- 172.21.15.12:0/1208671275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f674015d6c0 msgr2=0x7f67400abdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.463+0000 7f6746fb7700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f674015d6c0 0x7f67400abdf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6738014040 tx=0x7f6738004ad0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.463+0000 7f6746fb7700 1 -- 172.21.15.12:0/1208671275 shutdown_connections 2023-07-08T12:39:26.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.463+0000 7f6746fb7700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f672406ace0 0x7f672406d190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.463+0000 7f6746fb7700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f674015d6c0 0x7f67400abdf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.463+0000 7f6746fb7700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67400b1c70 0x7f6740157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.463+0000 7f6746fb7700 1 --2- 172.21.15.12:0/1208671275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67400ce1b0 0x7f67400ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.463+0000 7f6746fb7700 1 -- 172.21.15.12:0/1208671275 >> 172.21.15.12:0/1208671275 conn(0x7f67400cc290 msgr2=0x7f67400b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:26.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.463+0000 7f6746fb7700 1 -- 172.21.15.12:0/1208671275 shutdown_connections 2023-07-08T12:39:26.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.463+0000 7f6746fb7700 1 -- 172.21.15.12:0/1208671275 wait complete. 2023-07-08T12:39:26.478 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-07-08T12:39:26.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.602+0000 7f5f4218d700 1 Processor -- start 2023-07-08T12:39:26.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.602+0000 7f5f4218d700 1 -- start start 2023-07-08T12:39:26.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.603+0000 7f5f4218d700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5f3c0b2410 0x7f5f3c0b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:26.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.603+0000 7f5f4218d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5f3c0cee90 0x7f5f3c0cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:26.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.603+0000 7f5f4218d700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3c15a850 con 0x7f5f3c15a480 2023-07-08T12:39:26.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.603+0000 7f5f4218d700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3c0b2da0 con 0x7f5f3c0cee90 2023-07-08T12:39:26.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.603+0000 7f5f4218d700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3c0cde90 con 0x7f5f3c0b2410 2023-07-08T12:39:26.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.603+0000 7f5f3bfff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5f3c0cee90 0x7f5f3c0cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:26.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.603+0000 7f5f3bfff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5f3c0cee90 0x7f5f3c0cf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51564/0 (socket says 172.21.15.12:51564) 2023-07-08T12:39:26.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.604+0000 7f5f3bfff700 1 -- 172.21.15.12:0/2450504341 learned_addr learned my addr 172.21.15.12:0/2450504341 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:26.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.604+0000 7f5f3affd700 1 --2- 172.21.15.12:0/2450504341 >> v2:172.21.15.160:3300/0 conn(0x7f5f3c0b2410 0x7f5f3c0b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:26.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.604+0000 7f5f3bfff700 1 -- 172.21.15.12:0/2450504341 >> v2:172.21.15.160:3300/0 conn(0x7f5f3c0b2410 msgr2=0x7f5f3c0b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:26.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.604+0000 7f5f3bfff700 1 --2- 172.21.15.12:0/2450504341 >> v2:172.21.15.160:3300/0 conn(0x7f5f3c0b2410 0x7f5f3c0b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.604+0000 7f5f3bfff700 1 -- 172.21.15.12:0/2450504341 >> v1:172.21.15.160:6789/0 conn(0x7f5f3c15a480 legacy=0x7f5f3c0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.604+0000 7f5f3bfff700 1 -- 172.21.15.12:0/2450504341 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c0cd8d0 con 0x7f5f3c0cee90 2023-07-08T12:39:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.604+0000 7f5f3bfff700 1 --2- 172.21.15.12:0/2450504341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f3c0cee90 0x7f5f3c0cf2f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0d1a60 tx=0x7f5f3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f51b0c3bb50484 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.604+0000 7f5f3a7fc700 1 -- 172.21.15.12:0/2450504341 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400e070 con 0x7f5f3c0cee90 2023-07-08T12:39:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.604+0000 7f5f3a7fc700 1 -- 172.21.15.12:0/2450504341 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3400b910 con 0x7f5f3c0cee90 2023-07-08T12:39:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.604+0000 7f5f3a7fc700 1 -- 172.21.15.12:0/2450504341 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400bcc0 con 0x7f5f3c0cee90 2023-07-08T12:39:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.605+0000 7f5f4218d700 1 -- 172.21.15.12:0/2450504341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f3c0cee90 msgr2=0x7f5f3c0cf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.605+0000 7f5f4218d700 1 --2- 172.21.15.12:0/2450504341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f3c0cee90 0x7f5f3c0cf2f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0d1a60 tx=0x7f5f3400b330 comp rx=0 tx=0).stop 2023-07-08T12:39:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.605+0000 7f5f4218d700 1 -- 172.21.15.12:0/2450504341 shutdown_connections 2023-07-08T12:39:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.605+0000 7f5f4218d700 1 --2- 172.21.15.12:0/2450504341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f3c0cee90 0x7f5f3c0cf2f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.605+0000 7f5f4218d700 1 --2- 172.21.15.12:0/2450504341 >> v2:172.21.15.160:3300/0 conn(0x7f5f3c0b2410 0x7f5f3c0b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.605+0000 7f5f4218d700 1 -- 172.21.15.12:0/2450504341 >> 172.21.15.12:0/2450504341 conn(0x7f5f3c06ab60 msgr2=0x7f5f3c06e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.605+0000 7f5f4218d700 1 -- 172.21.15.12:0/2450504341 shutdown_connections 2023-07-08T12:39:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.605+0000 7f5f4218d700 1 -- 172.21.15.12:0/2450504341 wait complete. 2023-07-08T12:39:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.606+0000 7f5f4218d700 1 Processor -- start 2023-07-08T12:39:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.606+0000 7f5f4218d700 1 -- start start 2023-07-08T12:39:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f4218d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f3c0b2410 0x7f5f3c14c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f4218d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f3c0cee90 0x7f5f3c1f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f4218d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5f3c15a480 0x7f5f3c0c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f4218d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c0c87a0 con 0x7f5f3c0b2410 2023-07-08T12:39:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f4218d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c1f37c0 con 0x7f5f3c0cee90 2023-07-08T12:39:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f4218d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c1f3aa0 con 0x7f5f3c15a480 2023-07-08T12:39:26.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f3b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f3c0b2410 0x7f5f3c14c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:26.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f3b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f3c0b2410 0x7f5f3c14c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51566/0 (socket says 172.21.15.12:51566) 2023-07-08T12:39:26.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f3b7fe700 1 -- 172.21.15.12:0/1166423060 learned_addr learned my addr 172.21.15.12:0/1166423060 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:26.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f3affd700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f3c0cee90 0x7f5f3c1f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:26.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f3bfff700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5f3c15a480 0x7f5f3c0c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:26.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f3b7fe700 1 -- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5f3c15a480 msgr2=0x7f5f3c0c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f3b7fe700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5f3c15a480 0x7f5f3c0c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f3b7fe700 1 -- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f3c0cee90 msgr2=0x7f5f3c1f2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f3b7fe700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f3c0cee90 0x7f5f3c1f2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f3b7fe700 1 -- 172.21.15.12:0/1166423060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f34009b30 con 0x7f5f3c0b2410 2023-07-08T12:39:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f3bfff700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5f3c15a480 0x7f5f3c0c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.607+0000 7f5f3b7fe700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f3c0b2410 0x7f5f3c14c770 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00c4e0 tx=0x7f5f2c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.608+0000 7f5f23fff700 1 -- 172.21.15.12:0/1166423060 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0052d0 con 0x7f5f3c0b2410 2023-07-08T12:39:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.608+0000 7f5f23fff700 1 -- 172.21.15.12:0/1166423060 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c005430 con 0x7f5f3c0b2410 2023-07-08T12:39:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.608+0000 7f5f4218d700 1 -- 172.21.15.12:0/1166423060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3c1f3d80 con 0x7f5f3c0b2410 2023-07-08T12:39:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.608+0000 7f5f23fff700 1 -- 172.21.15.12:0/1166423060 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0038c0 con 0x7f5f3c0b2410 2023-07-08T12:39:26.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.608+0000 7f5f4218d700 1 -- 172.21.15.12:0/1166423060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c1f34b0 con 0x7f5f3c0b2410 2023-07-08T12:39:26.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.609+0000 7f5f4218d700 1 -- 172.21.15.12:0/1166423060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f3c0c8210 con 0x7f5f3c0b2410 2023-07-08T12:39:26.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.609+0000 7f5f23fff700 1 -- 172.21.15.12:0/1166423060 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f2c003a20 con 0x7f5f3c0b2410 2023-07-08T12:39:26.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.609+0000 7f5f23fff700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f5f2806ace0 0x7f5f2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:26.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.610+0000 7f5f23fff700 1 -- 172.21.15.12:0/1166423060 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5f2c0885e0 con 0x7f5f3c0b2410 2023-07-08T12:39:26.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.610+0000 7f5f3affd700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f5f2806ace0 0x7f5f2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:26.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.610+0000 7f5f3affd700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f5f2806ace0 0x7f5f2806d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0c81e0 tx=0x7f5f30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:26.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.614+0000 7f5f23fff700 1 -- 172.21.15.12:0/1166423060 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5f2c0538a0 con 0x7f5f3c0b2410 2023-07-08T12:39:26.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.899+0000 7f5f4218d700 1 -- 172.21.15.12:0/1166423060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f5f3c15a850 con 0x7f5f3c0b2410 2023-07-08T12:39:26.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.903+0000 7f5f23fff700 1 -- 172.21.15.12:0/1166423060 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5f3c15a850 con 0x7f5f3c0b2410 2023-07-08T12:39:26.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.905+0000 7f5f4218d700 1 -- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f5f2806ace0 msgr2=0x7f5f2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.905+0000 7f5f4218d700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f5f2806ace0 0x7f5f2806d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0c81e0 tx=0x7f5f30008040 comp rx=0 tx=0).stop 2023-07-08T12:39:26.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.905+0000 7f5f4218d700 1 -- 172.21.15.12:0/1166423060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f3c0b2410 msgr2=0x7f5f3c14c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:26.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.905+0000 7f5f4218d700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f3c0b2410 0x7f5f3c14c770 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00c4e0 tx=0x7f5f2c00c8a0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.905+0000 7f5f4218d700 1 -- 172.21.15.12:0/1166423060 shutdown_connections 2023-07-08T12:39:26.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.905+0000 7f5f4218d700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f5f2806ace0 0x7f5f2806d190 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.905+0000 7f5f4218d700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5f3c15a480 0x7f5f3c0c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.905+0000 7f5f4218d700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f3c0b2410 0x7f5f3c14c770 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.905+0000 7f5f4218d700 1 --2- 172.21.15.12:0/1166423060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f3c0cee90 0x7f5f3c1f2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:26.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.905+0000 7f5f4218d700 1 -- 172.21.15.12:0/1166423060 >> 172.21.15.12:0/1166423060 conn(0x7f5f3c06ab60 msgr2=0x7f5f3c0729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:26.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.905+0000 7f5f4218d700 1 -- 172.21.15.12:0/1166423060 shutdown_connections 2023-07-08T12:39:26.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:26.905+0000 7f5f4218d700 1 -- 172.21.15.12:0/1166423060 wait complete. 2023-07-08T12:39:26.920 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-07-08T12:39:27.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.044+0000 7f034357e700 1 Processor -- start 2023-07-08T12:39:27.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.044+0000 7f034357e700 1 -- start start 2023-07-08T12:39:27.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f034357e700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f033c15f7f0 0x7f033c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f034357e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f033c0b2b20 0x7f033c0ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f034357e700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033c0cd080 con 0x7f033c0ccbe0 2023-07-08T12:39:27.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f034357e700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033c0b3300 con 0x7f033c15f7f0 2023-07-08T12:39:27.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f034357e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0cd360 con 0x7f033c0b2b20 2023-07-08T12:39:27.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f0340b19700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f033c0ccbe0 0x7f033c0b2410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38972/0 (socket says 172.21.15.12:38972) 2023-07-08T12:39:27.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f0340b19700 1 -- 172.21.15.12:0/1249496209 learned_addr learned my addr 172.21.15.12:0/1249496209 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:27.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f0341b1b700 1 --2- 172.21.15.12:0/1249496209 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f033c0b2b20 0x7f033c0ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f034131a700 1 --2- 172.21.15.12:0/1249496209 >> v2:172.21.15.160:3300/0 conn(0x7f033c15f7f0 0x7f033c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f0333fff700 1 -- 172.21.15.12:0/1249496209 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750098296 0 0) 0x7f033c0cd080 con 0x7f033c0ccbe0 2023-07-08T12:39:27.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f0333fff700 1 -- 172.21.15.12:0/1249496209 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03280035b0 con 0x7f033c0ccbe0 2023-07-08T12:39:27.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f034131a700 1 -- 172.21.15.12:0/1249496209 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f033c0b2b20 msgr2=0x7f033c0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:27.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f034131a700 1 --2- 172.21.15.12:0/1249496209 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f033c0b2b20 0x7f033c0ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f034131a700 1 -- 172.21.15.12:0/1249496209 >> v1:172.21.15.12:6789/0 conn(0x7f033c0ccbe0 legacy=0x7f033c0b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f0333fff700 1 -- 172.21.15.12:0/1249496209 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1217144386 0 0) 0x7f03280035b0 con 0x7f033c0ccbe0 2023-07-08T12:39:27.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f034131a700 1 -- 172.21.15.12:0/1249496209 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c0cd640 con 0x7f033c15f7f0 2023-07-08T12:39:27.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.045+0000 7f0341b1b700 1 --2- 172.21.15.12:0/1249496209 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f033c0b2b20 0x7f033c0ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:39:27.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.046+0000 7f034131a700 1 --2- 172.21.15.12:0/1249496209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f033c15f7f0 0x7f033c0cafe0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f032c009840 tx=0x7f032c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beb9370f68b9a9ce server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:27.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.046+0000 7f0333fff700 1 -- 172.21.15.12:0/1249496209 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338001670 con 0x7f033c15f7f0 2023-07-08T12:39:27.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.046+0000 7f0333fff700 1 -- 172.21.15.12:0/1249496209 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c004030 con 0x7f033c15f7f0 2023-07-08T12:39:27.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.046+0000 7f0333fff700 1 -- 172.21.15.12:0/1249496209 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c0043c0 con 0x7f033c15f7f0 2023-07-08T12:39:27.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.046+0000 7f034357e700 1 -- 172.21.15.12:0/1249496209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f033c15f7f0 msgr2=0x7f033c0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.046+0000 7f034357e700 1 --2- 172.21.15.12:0/1249496209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f033c15f7f0 0x7f033c0cafe0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f032c009840 tx=0x7f032c009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:27.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.046+0000 7f034357e700 1 -- 172.21.15.12:0/1249496209 shutdown_connections 2023-07-08T12:39:27.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.046+0000 7f034357e700 1 --2- 172.21.15.12:0/1249496209 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f033c0b2b20 0x7f033c0ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.046+0000 7f034357e700 1 --2- 172.21.15.12:0/1249496209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f033c15f7f0 0x7f033c0cafe0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.046+0000 7f034357e700 1 -- 172.21.15.12:0/1249496209 >> 172.21.15.12:0/1249496209 conn(0x7f033c06ab60 msgr2=0x7f033c06e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:27.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.046+0000 7f034357e700 1 -- 172.21.15.12:0/1249496209 shutdown_connections 2023-07-08T12:39:27.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.046+0000 7f034357e700 1 -- 172.21.15.12:0/1249496209 wait complete. 2023-07-08T12:39:27.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.048+0000 7f034357e700 1 Processor -- start 2023-07-08T12:39:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.048+0000 7f034357e700 1 -- start start 2023-07-08T12:39:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f034357e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f033c0b2b20 0x7f033c0b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f034357e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f033c0ccbe0 0x7f033c0c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f034357e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f033c15f7f0 0x7f033c0c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f034357e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0b4110 con 0x7f033c0ccbe0 2023-07-08T12:39:27.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f034357e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0c8790 con 0x7f033c15f7f0 2023-07-08T12:39:27.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f034357e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0b43d0 con 0x7f033c0b2b20 2023-07-08T12:39:27.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f0340b19700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f033c0ccbe0 0x7f033c0c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f0340b19700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f033c0ccbe0 0x7f033c0c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51582/0 (socket says 172.21.15.12:51582) 2023-07-08T12:39:27.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f0340b19700 1 -- 172.21.15.12:0/3875973842 learned_addr learned my addr 172.21.15.12:0/3875973842 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:27.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f0341b1b700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f033c15f7f0 0x7f033c0c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f034131a700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f033c0b2b20 0x7f033c0b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f0340b19700 1 -- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f033c0b2b20 msgr2=0x7f033c0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f0340b19700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f033c0b2b20 0x7f033c0b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f0340b19700 1 -- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f033c15f7f0 msgr2=0x7f033c0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f0340b19700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f033c15f7f0 0x7f033c0c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f0340b19700 1 -- 172.21.15.12:0/3875973842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0334009a70 con 0x7f033c0ccbe0 2023-07-08T12:39:27.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f0341b1b700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f033c15f7f0 0x7f033c0c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:27.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.049+0000 7f034131a700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f033c0b2b20 0x7f033c0b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:27.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.050+0000 7f0340b19700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f033c0ccbe0 0x7f033c0c5c20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f033800c780 tx=0x7f033800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:27.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.050+0000 7f0331ffb700 1 -- 172.21.15.12:0/3875973842 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338004c80 con 0x7f033c0ccbe0 2023-07-08T12:39:27.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.050+0000 7f0331ffb700 1 -- 172.21.15.12:0/3875973842 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0338004de0 con 0x7f033c0ccbe0 2023-07-08T12:39:27.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.050+0000 7f034357e700 1 -- 172.21.15.12:0/3875973842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032c0094a0 con 0x7f033c0ccbe0 2023-07-08T12:39:27.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.050+0000 7f0331ffb700 1 -- 172.21.15.12:0/3875973842 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380072d0 con 0x7f033c0ccbe0 2023-07-08T12:39:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.050+0000 7f034357e700 1 -- 172.21.15.12:0/3875973842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c0c6160 con 0x7f033c0ccbe0 2023-07-08T12:39:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.051+0000 7f034357e700 1 -- 172.21.15.12:0/3875973842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c0b48f0 con 0x7f033c0ccbe0 2023-07-08T12:39:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.051+0000 7f0331ffb700 1 -- 172.21.15.12:0/3875973842 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0338007430 con 0x7f033c0ccbe0 2023-07-08T12:39:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.052+0000 7f0331ffb700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f032006ace0 0x7f032006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.052+0000 7f0331ffb700 1 -- 172.21.15.12:0/3875973842 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f03380888f0 con 0x7f033c0ccbe0 2023-07-08T12:39:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.052+0000 7f034131a700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f032006ace0 0x7f032006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.052+0000 7f034131a700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f032006ace0 0x7f032006d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f032c0090b0 tx=0x7f032c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:27.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.056+0000 7f0331ffb700 1 -- 172.21.15.12:0/3875973842 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f033c0b48f0 con 0x7f033c0ccbe0 2023-07-08T12:39:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.338+0000 7f034357e700 1 -- 172.21.15.12:0/3875973842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f033c0b48f0 con 0x7f033c0ccbe0 2023-07-08T12:39:27.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.343+0000 7f0331ffb700 1 -- 172.21.15.12:0/3875973842 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f033c0b48f0 con 0x7f033c0ccbe0 2023-07-08T12:39:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.345+0000 7f034357e700 1 -- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f032006ace0 msgr2=0x7f032006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.345+0000 7f034357e700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f032006ace0 0x7f032006d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f032c0090b0 tx=0x7f032c008030 comp rx=0 tx=0).stop 2023-07-08T12:39:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.345+0000 7f034357e700 1 -- 172.21.15.12:0/3875973842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f033c0ccbe0 msgr2=0x7f033c0c5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.345+0000 7f034357e700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f033c0ccbe0 0x7f033c0c5c20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f033800c780 tx=0x7f033800cb40 comp rx=0 tx=0).stop 2023-07-08T12:39:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.345+0000 7f034357e700 1 -- 172.21.15.12:0/3875973842 shutdown_connections 2023-07-08T12:39:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.345+0000 7f034357e700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f032006ace0 0x7f032006d190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.345+0000 7f034357e700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f033c15f7f0 0x7f033c0c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.345+0000 7f034357e700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f033c0b2b20 0x7f033c0b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.345+0000 7f034357e700 1 --2- 172.21.15.12:0/3875973842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f033c0ccbe0 0x7f033c0c5c20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.345+0000 7f034357e700 1 -- 172.21.15.12:0/3875973842 >> 172.21.15.12:0/3875973842 conn(0x7f033c06ab60 msgr2=0x7f033c06b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.345+0000 7f034357e700 1 -- 172.21.15.12:0/3875973842 shutdown_connections 2023-07-08T12:39:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.345+0000 7f034357e700 1 -- 172.21.15.12:0/3875973842 wait complete. 2023-07-08T12:39:27.359 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-07-08T12:39:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.483+0000 7ff0018a9700 1 Processor -- start 2023-07-08T12:39:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.483+0000 7ff0018a9700 1 -- start start 2023-07-08T12:39:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7ff0018a9700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feffc15c460 0x7feffc0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7ff0018a9700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7feffc0d2640 0x7feffc0cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7ff0018a9700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feffc0cf720 0x7feffc0cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7ff0018a9700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feffc0cedd0 con 0x7feffc15c460 2023-07-08T12:39:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7ff0018a9700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feffc0d1790 con 0x7feffc0d2640 2023-07-08T12:39:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7ff0018a9700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feffc0cd950 con 0x7feffc0cf720 2023-07-08T12:39:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7feffaffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feffc15c460 0x7feffc0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7feffaffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feffc15c460 0x7feffc0b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51588/0 (socket says 172.21.15.12:51588) 2023-07-08T12:39:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7feffaffd700 1 -- 172.21.15.12:0/3700633530 learned_addr learned my addr 172.21.15.12:0/3700633530 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7feffb7fe700 1 --2- 172.21.15.12:0/3700633530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feffc0cf720 0x7feffc0cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7feffa7fc700 1 --2- 172.21.15.12:0/3700633530 >> v2:172.21.15.160:3300/0 conn(0x7feffc0d2640 0x7feffc0cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7feffaffd700 1 -- 172.21.15.12:0/3700633530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feffc0cf720 msgr2=0x7feffc0cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7feffaffd700 1 --2- 172.21.15.12:0/3700633530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feffc0cf720 0x7feffc0cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7feffaffd700 1 -- 172.21.15.12:0/3700633530 >> v2:172.21.15.160:3300/0 conn(0x7feffc0d2640 msgr2=0x7feffc0cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7feffaffd700 1 --2- 172.21.15.12:0/3700633530 >> v2:172.21.15.160:3300/0 conn(0x7feffc0d2640 0x7feffc0cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.484+0000 7feffaffd700 1 -- 172.21.15.12:0/3700633530 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc0d14b0 con 0x7feffc15c460 2023-07-08T12:39:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.485+0000 7feffaffd700 1 --2- 172.21.15.12:0/3700633530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feffc15c460 0x7feffc0b59d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7feffc15fd80 tx=0x7feff0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9861b4480bb5eea server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.485+0000 7feff9ffb700 1 -- 172.21.15.12:0/3700633530 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff000b870 con 0x7feffc15c460 2023-07-08T12:39:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.485+0000 7feff9ffb700 1 -- 172.21.15.12:0/3700633530 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff000b9d0 con 0x7feffc15c460 2023-07-08T12:39:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.485+0000 7feff9ffb700 1 -- 172.21.15.12:0/3700633530 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff000bd60 con 0x7feffc15c460 2023-07-08T12:39:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.485+0000 7ff0018a9700 1 -- 172.21.15.12:0/3700633530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feffc15c460 msgr2=0x7feffc0b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.485+0000 7ff0018a9700 1 --2- 172.21.15.12:0/3700633530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feffc15c460 0x7feffc0b59d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7feffc15fd80 tx=0x7feff0009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.485+0000 7ff0018a9700 1 -- 172.21.15.12:0/3700633530 shutdown_connections 2023-07-08T12:39:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.485+0000 7ff0018a9700 1 --2- 172.21.15.12:0/3700633530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feffc0cf720 0x7feffc0cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.485+0000 7ff0018a9700 1 --2- 172.21.15.12:0/3700633530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feffc15c460 0x7feffc0b59d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.485+0000 7ff0018a9700 1 --2- 172.21.15.12:0/3700633530 >> v2:172.21.15.160:3300/0 conn(0x7feffc0d2640 0x7feffc0cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.485+0000 7ff0018a9700 1 -- 172.21.15.12:0/3700633530 >> 172.21.15.12:0/3700633530 conn(0x7feffc071660 msgr2=0x7feffc071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.485+0000 7ff0018a9700 1 -- 172.21.15.12:0/3700633530 shutdown_connections 2023-07-08T12:39:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.485+0000 7ff0018a9700 1 -- 172.21.15.12:0/3700633530 wait complete. 2023-07-08T12:39:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.487+0000 7ff0018a9700 1 Processor -- start 2023-07-08T12:39:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.487+0000 7ff0018a9700 1 -- start start 2023-07-08T12:39:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.488+0000 7ff0018a9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feffc0cf720 0x7feffc1f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.488+0000 7ff0018a9700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feffc0d2640 0x7feffc151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.488+0000 7ff0018a9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feffc15c460 0x7feffc1e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.488+0000 7ff0018a9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc1e6470 con 0x7feffc15c460 2023-07-08T12:39:27.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.488+0000 7ff0018a9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc1e6b20 con 0x7feffc0cf720 2023-07-08T12:39:27.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.488+0000 7ff0018a9700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feffc1f67e0 con 0x7feffc0d2640 2023-07-08T12:39:27.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.488+0000 7feffb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feffc15c460 0x7feffc1e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.488+0000 7feffb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feffc15c460 0x7feffc1e6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51596/0 (socket says 172.21.15.12:51596) 2023-07-08T12:39:27.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.488+0000 7feffb7fe700 1 -- 172.21.15.12:0/3209580097 learned_addr learned my addr 172.21.15.12:0/3209580097 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:27.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.488+0000 7feffaffd700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feffc0cf720 0x7feffc1f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.488+0000 7feffa7fc700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feffc0d2640 0x7feffc151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.489+0000 7feffb7fe700 1 -- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feffc0d2640 msgr2=0x7feffc151990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.489+0000 7feffb7fe700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feffc0d2640 0x7feffc151990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.489+0000 7feffb7fe700 1 -- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feffc0cf720 msgr2=0x7feffc1f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.489+0000 7feffb7fe700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feffc0cf720 0x7feffc1f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.489+0000 7feffb7fe700 1 -- 172.21.15.12:0/3209580097 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff0009550 con 0x7feffc15c460 2023-07-08T12:39:27.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.489+0000 7feffb7fe700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feffc15c460 0x7feffc1e6fb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7feff400cbb0 tx=0x7feff400cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:27.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.489+0000 7fefe3fff700 1 -- 172.21.15.12:0/3209580097 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4003c20 con 0x7feffc15c460 2023-07-08T12:39:27.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.489+0000 7fefe3fff700 1 -- 172.21.15.12:0/3209580097 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff4003d80 con 0x7feffc15c460 2023-07-08T12:39:27.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.489+0000 7ff0018a9700 1 -- 172.21.15.12:0/3209580097 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feffc1e5eb0 con 0x7feffc15c460 2023-07-08T12:39:27.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.489+0000 7fefe3fff700 1 -- 172.21.15.12:0/3209580097 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff400a820 con 0x7feffc15c460 2023-07-08T12:39:27.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.489+0000 7ff0018a9700 1 -- 172.21.15.12:0/3209580097 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feffc1e67b0 con 0x7feffc15c460 2023-07-08T12:39:27.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.490+0000 7ff0018a9700 1 -- 172.21.15.12:0/3209580097 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feffc04ea60 con 0x7feffc15c460 2023-07-08T12:39:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.490+0000 7fefe3fff700 1 -- 172.21.15.12:0/3209580097 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feff400a980 con 0x7feffc15c460 2023-07-08T12:39:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.491+0000 7fefe3fff700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fefe806ace0 0x7fefe806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.491+0000 7fefe3fff700 1 -- 172.21.15.12:0/3209580097 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7feff4088cb0 con 0x7feffc15c460 2023-07-08T12:39:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.491+0000 7feffaffd700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fefe806ace0 0x7fefe806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.492+0000 7feffaffd700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fefe806ace0 0x7fefe806d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feff0014040 tx=0x7feff0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.495+0000 7fefe3fff700 1 -- 172.21.15.12:0/3209580097 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feff4054000 con 0x7feffc15c460 2023-07-08T12:39:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.772+0000 7ff0018a9700 1 -- 172.21.15.12:0/3209580097 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7feffc1e6c60 con 0x7feffc15c460 2023-07-08T12:39:27.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.777+0000 7fefe3fff700 1 -- 172.21.15.12:0/3209580097 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7feff401e0a0 con 0x7feffc15c460 2023-07-08T12:39:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.779+0000 7ff0018a9700 1 -- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fefe806ace0 msgr2=0x7fefe806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.779+0000 7ff0018a9700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fefe806ace0 0x7fefe806d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feff0014040 tx=0x7feff0008030 comp rx=0 tx=0).stop 2023-07-08T12:39:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.779+0000 7ff0018a9700 1 -- 172.21.15.12:0/3209580097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feffc15c460 msgr2=0x7feffc1e6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.779+0000 7ff0018a9700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feffc15c460 0x7feffc1e6fb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7feff400cbb0 tx=0x7feff400cec0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.779+0000 7ff0018a9700 1 -- 172.21.15.12:0/3209580097 shutdown_connections 2023-07-08T12:39:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.779+0000 7ff0018a9700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fefe806ace0 0x7fefe806d190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.779+0000 7ff0018a9700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feffc15c460 0x7feffc1e6fb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.779+0000 7ff0018a9700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feffc0cf720 0x7feffc1f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.779+0000 7ff0018a9700 1 --2- 172.21.15.12:0/3209580097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feffc0d2640 0x7feffc151990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.779+0000 7ff0018a9700 1 -- 172.21.15.12:0/3209580097 >> 172.21.15.12:0/3209580097 conn(0x7feffc071660 msgr2=0x7feffc0d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.780+0000 7ff0018a9700 1 -- 172.21.15.12:0/3209580097 shutdown_connections 2023-07-08T12:39:27.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.780+0000 7ff0018a9700 1 -- 172.21.15.12:0/3209580097 wait complete. 2023-07-08T12:39:27.794 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:39:27.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.917+0000 7f2cb3f1b700 1 Processor -- start 2023-07-08T12:39:27.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.917+0000 7f2cb3f1b700 1 -- start start 2023-07-08T12:39:27.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb3f1b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2cac15f7f0 0x7f2cac0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb3f1b700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2cac0ccbe0 0x7f2cac0b2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb3f1b700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cac0cd080 con 0x7f2cac0cb520 2023-07-08T12:39:27.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb3f1b700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac0b3300 con 0x7f2cac15f7f0 2023-07-08T12:39:27.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb3f1b700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac0cd360 con 0x7f2cac0ccbe0 2023-07-08T12:39:27.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb1cb7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2cac15f7f0 0x7f2cac0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb24b8700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2cac0cb520 0x7f2cac15ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39002/0 (socket says 172.21.15.12:39002) 2023-07-08T12:39:27.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb24b8700 1 -- 172.21.15.12:0/1414018857 learned_addr learned my addr 172.21.15.12:0/1414018857 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:27.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb14b6700 1 --2- 172.21.15.12:0/1414018857 >> v2:172.21.15.160:3300/0 conn(0x7f2cac0ccbe0 0x7f2cac0b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb0cb5700 1 -- 172.21.15.12:0/1414018857 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129999410 0 0) 0x7f2cac0cd080 con 0x7f2cac0cb520 2023-07-08T12:39:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb0cb5700 1 -- 172.21.15.12:0/1414018857 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c980035b0 con 0x7f2cac0cb520 2023-07-08T12:39:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb1cb7700 1 -- 172.21.15.12:0/1414018857 >> v2:172.21.15.160:3300/0 conn(0x7f2cac0ccbe0 msgr2=0x7f2cac0b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb1cb7700 1 --2- 172.21.15.12:0/1414018857 >> v2:172.21.15.160:3300/0 conn(0x7f2cac0ccbe0 0x7f2cac0b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.918+0000 7f2cb1cb7700 1 -- 172.21.15.12:0/1414018857 >> v1:172.21.15.12:6789/0 conn(0x7f2cac0cb520 legacy=0x7f2cac15ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.919+0000 7f2cb1cb7700 1 -- 172.21.15.12:0/1414018857 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac0cd640 con 0x7f2cac15f7f0 2023-07-08T12:39:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.919+0000 7f2cb1cb7700 1 --2- 172.21.15.12:0/1414018857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cac15f7f0 0x7f2cac0cafe0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009840 tx=0x7f2c9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c663e054f4adb38f server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.919+0000 7f2cb0cb5700 1 -- 172.21.15.12:0/1414018857 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c011830 con 0x7f2cac15f7f0 2023-07-08T12:39:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.919+0000 7f2cb0cb5700 1 -- 172.21.15.12:0/1414018857 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9c011990 con 0x7f2cac15f7f0 2023-07-08T12:39:27.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.919+0000 7f2cb0cb5700 1 -- 172.21.15.12:0/1414018857 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c011d20 con 0x7f2cac15f7f0 2023-07-08T12:39:27.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.919+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/1414018857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cac15f7f0 msgr2=0x7f2cac0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.919+0000 7f2cb3f1b700 1 --2- 172.21.15.12:0/1414018857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cac15f7f0 0x7f2cac0cafe0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009840 tx=0x7f2c9c009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:27.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.919+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/1414018857 shutdown_connections 2023-07-08T12:39:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.919+0000 7f2cb3f1b700 1 --2- 172.21.15.12:0/1414018857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cac15f7f0 0x7f2cac0cafe0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.919+0000 7f2cb3f1b700 1 --2- 172.21.15.12:0/1414018857 >> v2:172.21.15.160:3300/0 conn(0x7f2cac0ccbe0 0x7f2cac0b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.919+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/1414018857 >> 172.21.15.12:0/1414018857 conn(0x7f2cac06ab60 msgr2=0x7f2cac06e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.920+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/1414018857 shutdown_connections 2023-07-08T12:39:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.920+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/1414018857 wait complete. 2023-07-08T12:39:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.921+0000 7f2cb3f1b700 1 Processor -- start 2023-07-08T12:39:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.921+0000 7f2cb3f1b700 1 -- start start 2023-07-08T12:39:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb3f1b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2cac0cb520 0x7f2cac150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb3f1b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cac0ccbe0 0x7f2cac0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb3f1b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cac15f7f0 0x7f2cac0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb3f1b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0b6440 con 0x7f2cac0ccbe0 2023-07-08T12:39:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb3f1b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac1513e0 con 0x7f2cac15f7f0 2023-07-08T12:39:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb3f1b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0c8640 con 0x7f2cac0cb520 2023-07-08T12:39:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb14b6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cac0ccbe0 0x7f2cac0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb14b6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cac0ccbe0 0x7f2cac0cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51610/0 (socket says 172.21.15.12:51610) 2023-07-08T12:39:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb14b6700 1 -- 172.21.15.12:0/2599461161 learned_addr learned my addr 172.21.15.12:0/2599461161 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb24b8700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cac15f7f0 0x7f2cac0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb1cb7700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2cac0cb520 0x7f2cac150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb14b6700 1 -- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2cac0cb520 msgr2=0x7f2cac150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb14b6700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2cac0cb520 0x7f2cac150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb14b6700 1 -- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cac15f7f0 msgr2=0x7f2cac0cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb14b6700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cac15f7f0 0x7f2cac0cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb14b6700 1 -- 172.21.15.12:0/2599461161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c0094a0 con 0x7f2cac0ccbe0 2023-07-08T12:39:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb14b6700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cac0ccbe0 0x7f2cac0cecb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e7e0 tx=0x7f2ca800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2ca27fc700 1 -- 172.21.15.12:0/2599461161 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800c070 con 0x7f2cac0ccbe0 2023-07-08T12:39:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2ca27fc700 1 -- 172.21.15.12:0/2599461161 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca8013610 con 0x7f2cac0ccbe0 2023-07-08T12:39:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/2599461161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac0c8920 con 0x7f2cac0ccbe0 2023-07-08T12:39:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2ca27fc700 1 -- 172.21.15.12:0/2599461161 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8013980 con 0x7f2cac0ccbe0 2023-07-08T12:39:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.922+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/2599461161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac0c8f50 con 0x7f2cac0ccbe0 2023-07-08T12:39:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.923+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/2599461161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac04ea60 con 0x7f2cac0ccbe0 2023-07-08T12:39:27.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.924+0000 7f2ca27fc700 1 -- 172.21.15.12:0/2599461161 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2ca8013ae0 con 0x7f2cac0ccbe0 2023-07-08T12:39:27.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.924+0000 7f2ca27fc700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2c9006ace0 0x7f2c9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:27.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.924+0000 7f2ca27fc700 1 -- 172.21.15.12:0/2599461161 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f2ca8088fa0 con 0x7f2cac0ccbe0 2023-07-08T12:39:27.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.924+0000 7f2cb1cb7700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2c9006ace0 0x7f2c9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:27.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.925+0000 7f2cb1cb7700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2c9006ace0 0x7f2c9006d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0090b0 tx=0x7f2c9c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:27.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:27.928+0000 7f2ca27fc700 1 -- 172.21.15.12:0/2599461161 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2ca8054270 con 0x7f2cac0ccbe0 2023-07-08T12:39:28.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.129+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/2599461161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2cac158680 con 0x7f2cac0ccbe0 2023-07-08T12:39:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.130+0000 7f2ca27fc700 1 -- 172.21.15.12:0/2599461161 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+14832 (secure 0 0 0) 0x7f2ca8057890 con 0x7f2cac0ccbe0 2023-07-08T12:39:28.132 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:28.133 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:39:24.673311+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:39:28.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.132+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2c9006ace0 msgr2=0x7f2c9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.132+0000 7f2cb3f1b700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2c9006ace0 0x7f2c9006d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0090b0 tx=0x7f2c9c01c040 comp rx=0 tx=0).stop 2023-07-08T12:39:28.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.132+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/2599461161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cac0ccbe0 msgr2=0x7f2cac0cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.132+0000 7f2cb3f1b700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cac0ccbe0 0x7f2cac0cecb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e7e0 tx=0x7f2ca800eaf0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.132+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/2599461161 shutdown_connections 2023-07-08T12:39:28.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.132+0000 7f2cb3f1b700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f2c9006ace0 0x7f2c9006d190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.132+0000 7f2cb3f1b700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cac15f7f0 0x7f2cac0cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.132+0000 7f2cb3f1b700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2cac0cb520 0x7f2cac150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.132+0000 7f2cb3f1b700 1 --2- 172.21.15.12:0/2599461161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cac0ccbe0 0x7f2cac0cecb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.132+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/2599461161 >> 172.21.15.12:0/2599461161 conn(0x7f2cac06ab60 msgr2=0x7f2cac0ce540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.132+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/2599461161 shutdown_connections 2023-07-08T12:39:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.132+0000 7f2cb3f1b700 1 -- 172.21.15.12:0/2599461161 wait complete. 2023-07-08T12:39:28.147 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-08T12:37:56.195631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-08T12:38:01.187530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-08T12:39:28.148 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-08T12:39:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.273+0000 7fda3bf8d700 1 Processor -- start 2023-07-08T12:39:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.274+0000 7fda3bf8d700 1 -- start start 2023-07-08T12:39:28.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.274+0000 7fda3bf8d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fda340aeec0 0x7fda3406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.274+0000 7fda3bf8d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda3406ebe0 0x7fda340cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.274+0000 7fda3bf8d700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fda340cd9b0 0x7fda340cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.274+0000 7fda3bf8d700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3415d770 con 0x7fda340aeec0 2023-07-08T12:39:28.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.274+0000 7fda3bf8d700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda340cddd0 con 0x7fda340cd9b0 2023-07-08T12:39:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.274+0000 7fda3bf8d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3415cb60 con 0x7fda3406ebe0 2023-07-08T12:39:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.274+0000 7fda39d29700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fda340aeec0 0x7fda3406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda39d29700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fda340aeec0 0x7fda3406cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51616/0 (socket says 172.21.15.12:51616) 2023-07-08T12:39:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda39d29700 1 -- 172.21.15.12:0/893776418 learned_addr learned my addr 172.21.15.12:0/893776418 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda3a52a700 1 --2- 172.21.15.12:0/893776418 >> v2:172.21.15.160:3300/0 conn(0x7fda340cd9b0 0x7fda340cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda39528700 1 --2- 172.21.15.12:0/893776418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda3406ebe0 0x7fda340cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda39d29700 1 -- 172.21.15.12:0/893776418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda3406ebe0 msgr2=0x7fda340cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:28.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda39d29700 1 --2- 172.21.15.12:0/893776418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda3406ebe0 0x7fda340cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda39d29700 1 -- 172.21.15.12:0/893776418 >> v2:172.21.15.160:3300/0 conn(0x7fda340cd9b0 msgr2=0x7fda340cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:28.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda39d29700 1 --2- 172.21.15.12:0/893776418 >> v2:172.21.15.160:3300/0 conn(0x7fda340cd9b0 0x7fda340cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda39d29700 1 -- 172.21.15.12:0/893776418 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3415ce40 con 0x7fda340aeec0 2023-07-08T12:39:28.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda3a52a700 1 --2- 172.21.15.12:0/893776418 >> v2:172.21.15.160:3300/0 conn(0x7fda340cd9b0 0x7fda340cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:28.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda39d29700 1 --2- 172.21.15.12:0/893776418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda340aeec0 0x7fda3406cff0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fda340724b0 tx=0x7fda2400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ed6ddc2ddbb6e server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:28.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda38d27700 1 -- 172.21.15.12:0/893776418 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24005e30 con 0x7fda340aeec0 2023-07-08T12:39:28.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda38d27700 1 -- 172.21.15.12:0/893776418 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda24004030 con 0x7fda340aeec0 2023-07-08T12:39:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.275+0000 7fda38d27700 1 -- 172.21.15.12:0/893776418 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240043e0 con 0x7fda340aeec0 2023-07-08T12:39:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.276+0000 7fda3bf8d700 1 -- 172.21.15.12:0/893776418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda340aeec0 msgr2=0x7fda3406cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.276+0000 7fda3bf8d700 1 --2- 172.21.15.12:0/893776418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda340aeec0 0x7fda3406cff0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fda340724b0 tx=0x7fda2400be70 comp rx=0 tx=0).stop 2023-07-08T12:39:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.276+0000 7fda3bf8d700 1 -- 172.21.15.12:0/893776418 shutdown_connections 2023-07-08T12:39:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.276+0000 7fda3bf8d700 1 --2- 172.21.15.12:0/893776418 >> v2:172.21.15.160:3300/0 conn(0x7fda340cd9b0 0x7fda340cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.276+0000 7fda3bf8d700 1 --2- 172.21.15.12:0/893776418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda340aeec0 0x7fda3406cff0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.276+0000 7fda3bf8d700 1 --2- 172.21.15.12:0/893776418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda3406ebe0 0x7fda340cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.276+0000 7fda3bf8d700 1 -- 172.21.15.12:0/893776418 >> 172.21.15.12:0/893776418 conn(0x7fda34098b50 msgr2=0x7fda3406b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:28.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.276+0000 7fda3bf8d700 1 -- 172.21.15.12:0/893776418 shutdown_connections 2023-07-08T12:39:28.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.276+0000 7fda3bf8d700 1 -- 172.21.15.12:0/893776418 wait complete. 2023-07-08T12:39:28.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.277+0000 7fda3bf8d700 1 Processor -- start 2023-07-08T12:39:28.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.277+0000 7fda3bf8d700 1 -- start start 2023-07-08T12:39:28.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda3bf8d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda3406ebe0 0x7fda340c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda3bf8d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda340aeec0 0x7fda340c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda3bf8d700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda340cd9b0 0x7fda340b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda3bf8d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda340b4940 con 0x7fda3406ebe0 2023-07-08T12:39:28.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda3bf8d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda340c96d0 con 0x7fda340aeec0 2023-07-08T12:39:28.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda3bf8d700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda34159920 con 0x7fda340cd9b0 2023-07-08T12:39:28.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda39d29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda3406ebe0 0x7fda340c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda39d29700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda3406ebe0 0x7fda340c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51622/0 (socket says 172.21.15.12:51622) 2023-07-08T12:39:28.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda39d29700 1 -- 172.21.15.12:0/3826820270 learned_addr learned my addr 172.21.15.12:0/3826820270 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:28.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda39d29700 1 -- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda340cd9b0 msgr2=0x7fda340b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda39528700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda340aeec0 0x7fda340c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda3a52a700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda340cd9b0 0x7fda340b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda39d29700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda340cd9b0 0x7fda340b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda39d29700 1 -- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda340aeec0 msgr2=0x7fda340c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda39d29700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda340aeec0 0x7fda340c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda39d29700 1 -- 172.21.15.12:0/3826820270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2400b7a0 con 0x7fda3406ebe0 2023-07-08T12:39:28.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda39528700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda340aeec0 0x7fda340c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:28.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.278+0000 7fda3a52a700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda340cd9b0 0x7fda340b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:28.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.279+0000 7fda39d29700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda3406ebe0 0x7fda340c6870 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda2400b340 tx=0x7fda24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:28.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.279+0000 7fda2a7fc700 1 -- 172.21.15.12:0/3826820270 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2401e070 con 0x7fda3406ebe0 2023-07-08T12:39:28.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.279+0000 7fda2a7fc700 1 -- 172.21.15.12:0/3826820270 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda240038b0 con 0x7fda3406ebe0 2023-07-08T12:39:28.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.279+0000 7fda3bf8d700 1 -- 172.21.15.12:0/3826820270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda340c7450 con 0x7fda3406ebe0 2023-07-08T12:39:28.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.279+0000 7fda2a7fc700 1 -- 172.21.15.12:0/3826820270 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24003c60 con 0x7fda3406ebe0 2023-07-08T12:39:28.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.279+0000 7fda3bf8d700 1 -- 172.21.15.12:0/3826820270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda340c7a10 con 0x7fda3406ebe0 2023-07-08T12:39:28.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.280+0000 7fda3bf8d700 1 -- 172.21.15.12:0/3826820270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3404ea60 con 0x7fda3406ebe0 2023-07-08T12:39:28.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.280+0000 7fda2a7fc700 1 -- 172.21.15.12:0/3826820270 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda240193f0 con 0x7fda3406ebe0 2023-07-08T12:39:28.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.281+0000 7fda2a7fc700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fda1806ace0 0x7fda1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.281+0000 7fda2a7fc700 1 -- 172.21.15.12:0/3826820270 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fda2408a2a0 con 0x7fda3406ebe0 2023-07-08T12:39:28.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.281+0000 7fda39528700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fda1806ace0 0x7fda1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.281+0000 7fda39528700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fda1806ace0 0x7fda1806d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda300099e0 tx=0x7fda30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:28.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.285+0000 7fda2a7fc700 1 -- 172.21.15.12:0/3826820270 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fda24055570 con 0x7fda3406ebe0 2023-07-08T12:39:28.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.483+0000 7fda3bf8d700 1 -- 172.21.15.12:0/3826820270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda34066af0 con 0x7fda3406ebe0 2023-07-08T12:39:28.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.484+0000 7fda2a7fc700 1 -- 172.21.15.12:0/3826820270 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda24058b90 con 0x7fda3406ebe0 2023-07-08T12:39:28.486 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-07-08T12:39:28.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.486+0000 7fda3bf8d700 1 -- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fda1806ace0 msgr2=0x7fda1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.486+0000 7fda3bf8d700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fda1806ace0 0x7fda1806d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda300099e0 tx=0x7fda30008040 comp rx=0 tx=0).stop 2023-07-08T12:39:28.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.486+0000 7fda3bf8d700 1 -- 172.21.15.12:0/3826820270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda3406ebe0 msgr2=0x7fda340c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.486+0000 7fda3bf8d700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda3406ebe0 0x7fda340c6870 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda2400b340 tx=0x7fda24004750 comp rx=0 tx=0).stop 2023-07-08T12:39:28.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.486+0000 7fda3bf8d700 1 -- 172.21.15.12:0/3826820270 shutdown_connections 2023-07-08T12:39:28.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.486+0000 7fda3bf8d700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fda1806ace0 0x7fda1806d190 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.486+0000 7fda3bf8d700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda340cd9b0 0x7fda340b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.486+0000 7fda3bf8d700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda3406ebe0 0x7fda340c6870 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.486+0000 7fda3bf8d700 1 --2- 172.21.15.12:0/3826820270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda340aeec0 0x7fda340c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.486+0000 7fda3bf8d700 1 -- 172.21.15.12:0/3826820270 >> 172.21.15.12:0/3826820270 conn(0x7fda34098b50 msgr2=0x7fda3406d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:28.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.486+0000 7fda3bf8d700 1 -- 172.21.15.12:0/3826820270 shutdown_connections 2023-07-08T12:39:28.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.486+0000 7fda3bf8d700 1 -- 172.21.15.12:0/3826820270 wait complete. 2023-07-08T12:39:28.500 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-08T12:39:28.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.627+0000 7f51454aa700 1 Processor -- start 2023-07-08T12:39:28.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.627+0000 7f51454aa700 1 -- start start 2023-07-08T12:39:28.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.628+0000 7f51454aa700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f514006e0c0 0x7f51401608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.628+0000 7f51454aa700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5140160410 con 0x7f5140156a10 2023-07-08T12:39:28.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.628+0000 7f51454aa700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5140156de0 con 0x7f51400cc240 2023-07-08T12:39:28.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.628+0000 7f51454aa700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514006ec70 con 0x7f514006e0c0 2023-07-08T12:39:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.628+0000 7f513e7fc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f514006e0c0 0x7f51401608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.628+0000 7f513effd700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5140156a10 0x7f514015d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39026/0 (socket says 172.21.15.12:39026) 2023-07-08T12:39:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.628+0000 7f513effd700 1 -- 172.21.15.12:0/3848102462 learned_addr learned my addr 172.21.15.12:0/3848102462 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f513e7fc700 1 -- 172.21.15.12:0/3848102462 >> v1:172.21.15.160:6789/0 conn(0x7f51400cc240 legacy=0x7f51400ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:28.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f513e7fc700 1 -- 172.21.15.12:0/3848102462 >> v1:172.21.15.12:6789/0 conn(0x7f5140156a10 legacy=0x7f514015d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f513e7fc700 1 -- 172.21.15.12:0/3848102462 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514006e6b0 con 0x7f514006e0c0 2023-07-08T12:39:28.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f513dffb700 1 -- 172.21.15.12:0/3848102462 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310712768 0 0) 0x7f5140160410 con 0x7f5140156a10 2023-07-08T12:39:28.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f513e7fc700 1 --2- 172.21.15.12:0/3848102462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f514006e0c0 0x7f51401608a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5130009840 tx=0x7f5130009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31dd4386a3bbb49 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:28.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f513dffb700 1 -- 172.21.15.12:0/3848102462 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5130004230 con 0x7f514006e0c0 2023-07-08T12:39:28.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f513dffb700 1 -- 172.21.15.12:0/3848102462 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5130004390 con 0x7f514006e0c0 2023-07-08T12:39:28.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f513dffb700 1 -- 172.21.15.12:0/3848102462 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51300036a0 con 0x7f514006e0c0 2023-07-08T12:39:28.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f51454aa700 1 -- 172.21.15.12:0/3848102462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f514006e0c0 msgr2=0x7f51401608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f51454aa700 1 --2- 172.21.15.12:0/3848102462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f514006e0c0 0x7f51401608a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5130009840 tx=0x7f5130009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:28.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f51454aa700 1 -- 172.21.15.12:0/3848102462 shutdown_connections 2023-07-08T12:39:28.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f51454aa700 1 --2- 172.21.15.12:0/3848102462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f514006e0c0 0x7f51401608a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f51454aa700 1 -- 172.21.15.12:0/3848102462 >> 172.21.15.12:0/3848102462 conn(0x7f5140098b50 msgr2=0x7f514006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:28.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f51454aa700 1 -- 172.21.15.12:0/3848102462 shutdown_connections 2023-07-08T12:39:28.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.629+0000 7f51454aa700 1 -- 172.21.15.12:0/3848102462 wait complete. 2023-07-08T12:39:28.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.631+0000 7f51454aa700 1 Processor -- start 2023-07-08T12:39:28.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.631+0000 7f51454aa700 1 -- start start 2023-07-08T12:39:28.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f51454aa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f514006e0c0 0x7f5140159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f51454aa700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51400cc240 0x7f51400ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f51454aa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5140156a10 0x7f51400ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f51454aa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51400b3950 con 0x7f5140156a10 2023-07-08T12:39:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f51454aa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5140157d00 con 0x7f514006e0c0 2023-07-08T12:39:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f51454aa700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51400b3020 con 0x7f51400cc240 2023-07-08T12:39:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f513f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5140156a10 0x7f51400ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f513f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5140156a10 0x7f51400ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51640/0 (socket says 172.21.15.12:51640) 2023-07-08T12:39:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f513f7fe700 1 -- 172.21.15.12:0/2946765551 learned_addr learned my addr 172.21.15.12:0/2946765551 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f513effd700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f514006e0c0 0x7f5140159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f513e7fc700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51400cc240 0x7f51400ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f513f7fe700 1 -- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51400cc240 msgr2=0x7f51400ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f513f7fe700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51400cc240 0x7f51400ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f513f7fe700 1 -- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f514006e0c0 msgr2=0x7f5140159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f513f7fe700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f514006e0c0 0x7f5140159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f513f7fe700 1 -- 172.21.15.12:0/2946765551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300094a0 con 0x7f5140156a10 2023-07-08T12:39:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f513e7fc700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51400cc240 0x7f51400ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f513f7fe700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5140156a10 0x7f51400ce8b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f513800b970 tx=0x7f513800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f51277fe700 1 -- 172.21.15.12:0/2946765551 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513800c9b0 con 0x7f5140156a10 2023-07-08T12:39:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.632+0000 7f51277fe700 1 -- 172.21.15.12:0/2946765551 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513800cb10 con 0x7f5140156a10 2023-07-08T12:39:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.633+0000 7f51454aa700 1 -- 172.21.15.12:0/2946765551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51400b3300 con 0x7f5140156a10 2023-07-08T12:39:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.633+0000 7f51277fe700 1 -- 172.21.15.12:0/2946765551 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138014440 con 0x7f5140156a10 2023-07-08T12:39:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.633+0000 7f51454aa700 1 -- 172.21.15.12:0/2946765551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514015a1f0 con 0x7f5140156a10 2023-07-08T12:39:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.633+0000 7f51454aa700 1 -- 172.21.15.12:0/2946765551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51400b3ac0 con 0x7f5140156a10 2023-07-08T12:39:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.634+0000 7f51277fe700 1 -- 172.21.15.12:0/2946765551 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f513801d3b0 con 0x7f5140156a10 2023-07-08T12:39:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.634+0000 7f51277fe700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f512c06ace0 0x7f512c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.634+0000 7f51277fe700 1 -- 172.21.15.12:0/2946765551 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5138088f20 con 0x7f5140156a10 2023-07-08T12:39:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.634+0000 7f513effd700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f512c06ace0 0x7f512c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.635+0000 7f513effd700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f512c06ace0 0x7f512c06d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5134007fd0 tx=0x7f5134009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.639+0000 7f51277fe700 1 -- 172.21.15.12:0/2946765551 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f51380541e0 con 0x7f5140156a10 2023-07-08T12:39:28.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.837+0000 7f51454aa700 1 -- 172.21.15.12:0/2946765551 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5140160070 con 0x7f5140156a10 2023-07-08T12:39:28.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.837+0000 7f51277fe700 1 -- 172.21.15.12:0/2946765551 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v38) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5138019020 con 0x7f5140156a10 2023-07-08T12:39:28.840 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2023-07-08T12:39:28.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.839+0000 7f51454aa700 1 -- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f512c06ace0 msgr2=0x7f512c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.839+0000 7f51454aa700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f512c06ace0 0x7f512c06d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5134007fd0 tx=0x7f5134009040 comp rx=0 tx=0).stop 2023-07-08T12:39:28.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.839+0000 7f51454aa700 1 -- 172.21.15.12:0/2946765551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5140156a10 msgr2=0x7f51400ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.839+0000 7f51454aa700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5140156a10 0x7f51400ce8b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f513800b970 tx=0x7f513800bd30 comp rx=0 tx=0).stop 2023-07-08T12:39:28.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.839+0000 7f51454aa700 1 -- 172.21.15.12:0/2946765551 shutdown_connections 2023-07-08T12:39:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.839+0000 7f51454aa700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f512c06ace0 0x7f512c06d190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.839+0000 7f51454aa700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5140156a10 0x7f51400ce8b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.839+0000 7f51454aa700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f514006e0c0 0x7f5140159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.839+0000 7f51454aa700 1 --2- 172.21.15.12:0/2946765551 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51400cc240 0x7f51400ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.839+0000 7f51454aa700 1 -- 172.21.15.12:0/2946765551 >> 172.21.15.12:0/2946765551 conn(0x7f5140098b50 msgr2=0x7f51400b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.839+0000 7f51454aa700 1 -- 172.21.15.12:0/2946765551 shutdown_connections 2023-07-08T12:39:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.840+0000 7f51454aa700 1 -- 172.21.15.12:0/2946765551 wait complete. 2023-07-08T12:39:28.854 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-08T12:39:28.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.977+0000 7f94099f7700 1 Processor -- start 2023-07-08T12:39:28.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.977+0000 7f94099f7700 1 -- start start 2023-07-08T12:39:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.977+0000 7f94099f7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f940415c1e0 0x7f94040caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.977+0000 7f94099f7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94040cb470 0x7f940406ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.977+0000 7f94099f7700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f940406bf90 0x7f940415fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.977+0000 7f94099f7700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94040d06d0 con 0x7f940415c1e0 2023-07-08T12:39:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.977+0000 7f94099f7700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94040ce6e0 con 0x7f940406bf90 2023-07-08T12:39:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.977+0000 7f94099f7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94040ceb70 con 0x7f94040cb470 2023-07-08T12:39:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f9402ffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f940415c1e0 0x7f94040caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f9402ffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f940415c1e0 0x7f94040caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51644/0 (socket says 172.21.15.12:51644) 2023-07-08T12:39:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f9402ffd700 1 -- 172.21.15.12:0/665641194 learned_addr learned my addr 172.21.15.12:0/665641194 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f94027fc700 1 --2- 172.21.15.12:0/665641194 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94040cb470 0x7f940406ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f94037fe700 1 --2- 172.21.15.12:0/665641194 >> v2:172.21.15.160:3300/0 conn(0x7f940406bf90 0x7f940415fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f9402ffd700 1 -- 172.21.15.12:0/665641194 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94040cb470 msgr2=0x7f940406ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f9402ffd700 1 --2- 172.21.15.12:0/665641194 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94040cb470 0x7f940406ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f9402ffd700 1 -- 172.21.15.12:0/665641194 >> v2:172.21.15.160:3300/0 conn(0x7f940406bf90 msgr2=0x7f940415fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f9402ffd700 1 --2- 172.21.15.12:0/665641194 >> v2:172.21.15.160:3300/0 conn(0x7f940406bf90 0x7f940415fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f9402ffd700 1 -- 172.21.15.12:0/665641194 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9404070b60 con 0x7f940415c1e0 2023-07-08T12:39:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f9402ffd700 1 --2- 172.21.15.12:0/665641194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f940415c1e0 0x7f94040caf30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f93f80098f0 tx=0x7f93f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1023df9985ca8114 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f9401ffb700 1 -- 172.21.15.12:0/665641194 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800b860 con 0x7f940415c1e0 2023-07-08T12:39:28.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f9401ffb700 1 -- 172.21.15.12:0/665641194 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f800b9c0 con 0x7f940415c1e0 2023-07-08T12:39:28.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.978+0000 7f9401ffb700 1 -- 172.21.15.12:0/665641194 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800bd50 con 0x7f940415c1e0 2023-07-08T12:39:28.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.979+0000 7f94099f7700 1 -- 172.21.15.12:0/665641194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f940415c1e0 msgr2=0x7f94040caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.979+0000 7f94099f7700 1 --2- 172.21.15.12:0/665641194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f940415c1e0 0x7f94040caf30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f93f80098f0 tx=0x7f93f8009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:28.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.979+0000 7f94099f7700 1 -- 172.21.15.12:0/665641194 shutdown_connections 2023-07-08T12:39:28.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.979+0000 7f94099f7700 1 --2- 172.21.15.12:0/665641194 >> v2:172.21.15.160:3300/0 conn(0x7f940406bf90 0x7f940415fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.979+0000 7f94099f7700 1 --2- 172.21.15.12:0/665641194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f940415c1e0 0x7f94040caf30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.979+0000 7f94099f7700 1 --2- 172.21.15.12:0/665641194 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94040cb470 0x7f940406ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.979+0000 7f94099f7700 1 -- 172.21.15.12:0/665641194 >> 172.21.15.12:0/665641194 conn(0x7f940406e550 msgr2=0x7f940406e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.979+0000 7f94099f7700 1 -- 172.21.15.12:0/665641194 shutdown_connections 2023-07-08T12:39:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.979+0000 7f94099f7700 1 -- 172.21.15.12:0/665641194 wait complete. 2023-07-08T12:39:28.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.980+0000 7f94099f7700 1 Processor -- start 2023-07-08T12:39:28.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.980+0000 7f94099f7700 1 -- start start 2023-07-08T12:39:28.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.981+0000 7f94099f7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940406bf90 0x7f94041f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.981+0000 7f94099f7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94040cb470 0x7f940414dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.981+0000 7f94099f7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f940415c1e0 0x7f94041e8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.981+0000 7f94099f7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94041f2630 con 0x7f94040cb470 2023-07-08T12:39:28.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.981+0000 7f94099f7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94040c8e10 con 0x7f940406bf90 2023-07-08T12:39:28.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.981+0000 7f94099f7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94040c90f0 con 0x7f940415c1e0 2023-07-08T12:39:28.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.981+0000 7f94027fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94040cb470 0x7f940414dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.981+0000 7f94027fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94040cb470 0x7f940414dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51652/0 (socket says 172.21.15.12:51652) 2023-07-08T12:39:28.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.981+0000 7f94027fc700 1 -- 172.21.15.12:0/746134556 learned_addr learned my addr 172.21.15.12:0/746134556 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:28.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.981+0000 7f94037fe700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f940415c1e0 0x7f94041e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.981+0000 7f9402ffd700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940406bf90 0x7f94041f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.981+0000 7f94027fc700 1 -- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f940415c1e0 msgr2=0x7f94041e8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.981+0000 7f94027fc700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f940415c1e0 0x7f94041e8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.982+0000 7f94027fc700 1 -- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940406bf90 msgr2=0x7f94041f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:28.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.982+0000 7f94027fc700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940406bf90 0x7f94041f20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:28.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.982+0000 7f94027fc700 1 -- 172.21.15.12:0/746134556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f8009550 con 0x7f94040cb470 2023-07-08T12:39:28.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.982+0000 7f94027fc700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94040cb470 0x7f940414dcc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f93f400e810 tx=0x7f93f400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:28.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.982+0000 7f93ebfff700 1 -- 172.21.15.12:0/746134556 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f400c070 con 0x7f94040cb470 2023-07-08T12:39:28.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.982+0000 7f93ebfff700 1 -- 172.21.15.12:0/746134556 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f4013610 con 0x7f94040cb470 2023-07-08T12:39:28.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.982+0000 7f94099f7700 1 -- 172.21.15.12:0/746134556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94040c93d0 con 0x7f94040cb470 2023-07-08T12:39:28.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.982+0000 7f93ebfff700 1 -- 172.21.15.12:0/746134556 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40139c0 con 0x7f94040cb470 2023-07-08T12:39:28.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.982+0000 7f94099f7700 1 -- 172.21.15.12:0/746134556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94040c9990 con 0x7f94040cb470 2023-07-08T12:39:28.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.983+0000 7f94099f7700 1 -- 172.21.15.12:0/746134556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940404ea60 con 0x7f94040cb470 2023-07-08T12:39:28.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.983+0000 7f93ebfff700 1 -- 172.21.15.12:0/746134556 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f93f4013b20 con 0x7f94040cb470 2023-07-08T12:39:28.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.984+0000 7f93ebfff700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f93f006ace0 0x7f93f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:28.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.984+0000 7f93ebfff700 1 -- 172.21.15.12:0/746134556 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f93f4089030 con 0x7f94040cb470 2023-07-08T12:39:28.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.984+0000 7f9402ffd700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f93f006ace0 0x7f93f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:28.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.984+0000 7f9402ffd700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f93f006ace0 0x7f93f006d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f93f8014040 tx=0x7f93f8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:28.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:28.988+0000 7f93ebfff700 1 -- 172.21.15.12:0/746134556 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f93f4057cf0 con 0x7f94040cb470 2023-07-08T12:39:29.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.189+0000 7f94099f7700 1 -- 172.21.15.12:0/746134556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9404070710 con 0x7f94040cb470 2023-07-08T12:39:29.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.189+0000 7f93ebfff700 1 -- 172.21.15.12:0/746134556 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v38) v1 ==== 84+0+126 (secure 0 0 0) 0x7f93f4057880 con 0x7f94040cb470 2023-07-08T12:39:29.192 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:29.192 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2023-07-08T12:39:29.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:29.192 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 1, 2023-07-08T12:39:29.193 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": ".mgr" 2023-07-08T12:39:29.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-07-08T12:39:29.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-07-08T12:39:29.193 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 2, 2023-07-08T12:39:29.193 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": "rbd" 2023-07-08T12:39:29.193 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-07-08T12:39:29.193 INFO:teuthology.orchestra.run.smithi012.stdout:] 2023-07-08T12:39:29.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.191+0000 7f94099f7700 1 -- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f93f006ace0 msgr2=0x7f93f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.191+0000 7f94099f7700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f93f006ace0 0x7f93f006d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f93f8014040 tx=0x7f93f8008030 comp rx=0 tx=0).stop 2023-07-08T12:39:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.191+0000 7f94099f7700 1 -- 172.21.15.12:0/746134556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94040cb470 msgr2=0x7f940414dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.191+0000 7f94099f7700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94040cb470 0x7f940414dcc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f93f400e810 tx=0x7f93f400eb20 comp rx=0 tx=0).stop 2023-07-08T12:39:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.191+0000 7f94099f7700 1 -- 172.21.15.12:0/746134556 shutdown_connections 2023-07-08T12:39:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.191+0000 7f94099f7700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f93f006ace0 0x7f93f006d190 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.191+0000 7f94099f7700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f940415c1e0 0x7f94041e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.191+0000 7f94099f7700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f940406bf90 0x7f94041f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.191+0000 7f94099f7700 1 --2- 172.21.15.12:0/746134556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94040cb470 0x7f940414dcc0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.192+0000 7f94099f7700 1 -- 172.21.15.12:0/746134556 >> 172.21.15.12:0/746134556 conn(0x7f940406e550 msgr2=0x7f94040afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:29.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.192+0000 7f94099f7700 1 -- 172.21.15.12:0/746134556 shutdown_connections 2023-07-08T12:39:29.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.192+0000 7f94099f7700 1 -- 172.21.15.12:0/746134556 wait complete. 2023-07-08T12:39:29.206 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-08T12:39:29.207 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-08T12:39:29.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.334+0000 7fe5c7b12700 1 Processor -- start 2023-07-08T12:39:29.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.334+0000 7fe5c7b12700 1 -- start start 2023-07-08T12:39:29.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.335+0000 7fe5c7b12700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5c00cd340 0x7fe5c00caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:29.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.335+0000 7fe5c7b12700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe5c00ccae0 0x7fe5c00cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:29.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.335+0000 7fe5c7b12700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c00cf8a0 con 0x7fe5c00b3780 2023-07-08T12:39:29.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.335+0000 7fe5c7b12700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c00ccf50 con 0x7fe5c00ccae0 2023-07-08T12:39:29.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.335+0000 7fe5c7b12700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c00cefb0 con 0x7fe5c00cd340 2023-07-08T12:39:29.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.336+0000 7fe5c50ad700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe5c00ccae0 0x7fe5c00cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:29.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.336+0000 7fe5c58ae700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5c00cd340 0x7fe5c00caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:29.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.336+0000 7fe5c50ad700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe5c00ccae0 0x7fe5c00cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.12:50404/0 (socket says 172.21.15.12:50404) 2023-07-08T12:39:29.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.336+0000 7fe5c50ad700 1 -- 172.21.15.12:0/511254331 learned_addr learned my addr 172.21.15.12:0/511254331 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:29.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.336+0000 7fe5c50ad700 1 -- 172.21.15.12:0/511254331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5c00cd340 msgr2=0x7fe5c00caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:29.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.336+0000 7fe5c50ad700 1 --2- 172.21.15.12:0/511254331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5c00cd340 0x7fe5c00caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.336+0000 7fe5c50ad700 1 -- 172.21.15.12:0/511254331 >> v1:172.21.15.160:6789/0 conn(0x7fe5c00b3780 legacy=0x7fe5c00cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:29.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.336+0000 7fe5c50ad700 1 -- 172.21.15.12:0/511254331 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c00cdc50 con 0x7fe5c00ccae0 2023-07-08T12:39:29.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.336+0000 7fe5c50ad700 1 --2- 172.21.15.12:0/511254331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c00ccae0 0x7fe5c00cd710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0098f0 tx=0x7fe5bc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed5c446dd3b3d768 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:29.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.336+0000 7fe5c48ac700 1 -- 172.21.15.12:0/511254331 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc011830 con 0x7fe5c00ccae0 2023-07-08T12:39:29.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.336+0000 7fe5c48ac700 1 -- 172.21.15.12:0/511254331 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc011990 con 0x7fe5c00ccae0 2023-07-08T12:39:29.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.336+0000 7fe5c48ac700 1 -- 172.21.15.12:0/511254331 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc011d20 con 0x7fe5c00ccae0 2023-07-08T12:39:29.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.337+0000 7fe5c7b12700 1 -- 172.21.15.12:0/511254331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c00ccae0 msgr2=0x7fe5c00cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:29.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.337+0000 7fe5c7b12700 1 --2- 172.21.15.12:0/511254331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c00ccae0 0x7fe5c00cd710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0098f0 tx=0x7fe5bc009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:29.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.337+0000 7fe5c7b12700 1 -- 172.21.15.12:0/511254331 shutdown_connections 2023-07-08T12:39:29.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.337+0000 7fe5c7b12700 1 --2- 172.21.15.12:0/511254331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5c00cd340 0x7fe5c00caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.337+0000 7fe5c7b12700 1 --2- 172.21.15.12:0/511254331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c00ccae0 0x7fe5c00cd710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.337+0000 7fe5c7b12700 1 -- 172.21.15.12:0/511254331 >> 172.21.15.12:0/511254331 conn(0x7fe5c006e670 msgr2=0x7fe5c006ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:29.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.337+0000 7fe5c7b12700 1 -- 172.21.15.12:0/511254331 shutdown_connections 2023-07-08T12:39:29.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.337+0000 7fe5c7b12700 1 -- 172.21.15.12:0/511254331 wait complete. 2023-07-08T12:39:29.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.338+0000 7fe5c7b12700 1 Processor -- start 2023-07-08T12:39:29.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.338+0000 7fe5c7b12700 1 -- start start 2023-07-08T12:39:29.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c7b12700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c00b3780 0x7fe5c006ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:29.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c7b12700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5c00ccae0 0x7fe5c014be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:29.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c7b12700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c00cd340 0x7fe5c006bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:29.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c7b12700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c006d0c0 con 0x7fe5c00cd340 2023-07-08T12:39:29.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c7b12700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c014c350 con 0x7fe5c00b3780 2023-07-08T12:39:29.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c7b12700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c006bb10 con 0x7fe5c00ccae0 2023-07-08T12:39:29.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c60af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c00cd340 0x7fe5c006bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:29.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c60af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c00cd340 0x7fe5c006bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51668/0 (socket says 172.21.15.12:51668) 2023-07-08T12:39:29.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c60af700 1 -- 172.21.15.12:0/2888952462 learned_addr learned my addr 172.21.15.12:0/2888952462 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:29.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c58ae700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c00b3780 0x7fe5c006ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:29.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c50ad700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5c00ccae0 0x7fe5c014be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:29.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c60af700 1 -- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5c00ccae0 msgr2=0x7fe5c014be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:29.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c60af700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5c00ccae0 0x7fe5c014be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c60af700 1 -- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c00b3780 msgr2=0x7fe5c006ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:29.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.339+0000 7fe5c60af700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c00b3780 0x7fe5c006ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.340+0000 7fe5c60af700 1 -- 172.21.15.12:0/2888952462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc009550 con 0x7fe5c00cd340 2023-07-08T12:39:29.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.340+0000 7fe5c60af700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c00cd340 0x7fe5c006bfa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800cb50 tx=0x7fe5b800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:29.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.340+0000 7fe5b67fc700 1 -- 172.21.15.12:0/2888952462 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8003c00 con 0x7fe5c00cd340 2023-07-08T12:39:29.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.340+0000 7fe5b67fc700 1 -- 172.21.15.12:0/2888952462 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b8003d60 con 0x7fe5c00cd340 2023-07-08T12:39:29.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.340+0000 7fe5c7b12700 1 -- 172.21.15.12:0/2888952462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c014f6d0 con 0x7fe5c00cd340 2023-07-08T12:39:29.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.340+0000 7fe5b67fc700 1 -- 172.21.15.12:0/2888952462 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b800aa20 con 0x7fe5c00cd340 2023-07-08T12:39:29.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.340+0000 7fe5c7b12700 1 -- 172.21.15.12:0/2888952462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c006b4c0 con 0x7fe5c00cd340 2023-07-08T12:39:29.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.341+0000 7fe5c7b12700 1 -- 172.21.15.12:0/2888952462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c00b3b50 con 0x7fe5c00cd340 2023-07-08T12:39:29.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.341+0000 7fe5b67fc700 1 -- 172.21.15.12:0/2888952462 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe5b800ab80 con 0x7fe5c00cd340 2023-07-08T12:39:29.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.342+0000 7fe5b67fc700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe5a406ace0 0x7fe5a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:29.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.342+0000 7fe5b67fc700 1 -- 172.21.15.12:0/2888952462 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe5b80891f0 con 0x7fe5c00cd340 2023-07-08T12:39:29.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.342+0000 7fe5c58ae700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe5a406ace0 0x7fe5a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:29.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.342+0000 7fe5c58ae700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe5a406ace0 0x7fe5a406d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009be0 tx=0x7fe5b0009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:29.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.346+0000 7fe5b67fc700 1 -- 172.21.15.12:0/2888952462 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe5b8054540 con 0x7fe5c00cd340 2023-07-08T12:39:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.547+0000 7fe5c7b12700 1 -- 172.21.15.12:0/2888952462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe5c006bc80 con 0x7fe5c00cd340 2023-07-08T12:39:29.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.786+0000 7fe5b67fc700 1 -- 172.21.15.12:0/2888952462 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v39) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe5b8057b60 con 0x7fe5c00cd340 2023-07-08T12:39:29.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.789+0000 7fe5c7b12700 1 -- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe5a406ace0 msgr2=0x7fe5a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:29.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.789+0000 7fe5c7b12700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe5a406ace0 0x7fe5a406d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009be0 tx=0x7fe5b0009210 comp rx=0 tx=0).stop 2023-07-08T12:39:29.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.789+0000 7fe5c7b12700 1 -- 172.21.15.12:0/2888952462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c00cd340 msgr2=0x7fe5c006bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:29.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.789+0000 7fe5c7b12700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c00cd340 0x7fe5c006bfa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800cb50 tx=0x7fe5b800ce60 comp rx=0 tx=0).stop 2023-07-08T12:39:29.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.789+0000 7fe5c7b12700 1 -- 172.21.15.12:0/2888952462 shutdown_connections 2023-07-08T12:39:29.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.789+0000 7fe5c7b12700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fe5a406ace0 0x7fe5a406d190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.789+0000 7fe5c7b12700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5c00cd340 0x7fe5c006bfa0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.789+0000 7fe5c7b12700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5c00b3780 0x7fe5c006ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.789+0000 7fe5c7b12700 1 --2- 172.21.15.12:0/2888952462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5c00ccae0 0x7fe5c014be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.789+0000 7fe5c7b12700 1 -- 172.21.15.12:0/2888952462 >> 172.21.15.12:0/2888952462 conn(0x7fe5c006e670 msgr2=0x7fe5c00b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:29.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.789+0000 7fe5c7b12700 1 -- 172.21.15.12:0/2888952462 shutdown_connections 2023-07-08T12:39:29.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.790+0000 7fe5c7b12700 1 -- 172.21.15.12:0/2888952462 wait complete. 2023-07-08T12:39:29.793 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' created 2023-07-08T12:39:29.807 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-08T12:39:29.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.937+0000 7f70cff0e700 1 Processor -- start 2023-07-08T12:39:29.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.938+0000 7f70cff0e700 1 -- start start 2023-07-08T12:39:29.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.938+0000 7f70cff0e700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70c8156a10 0x7f70c806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:29.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.938+0000 7f70cff0e700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f70c80cc240 0x7f70c80cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:29.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.938+0000 7f70cff0e700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c80ccbe0 con 0x7f70c815d390 2023-07-08T12:39:29.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.938+0000 7f70cff0e700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c8160410 con 0x7f70c8156a10 2023-07-08T12:39:29.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.938+0000 7f70cff0e700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c8156de0 con 0x7f70c80cc240 2023-07-08T12:39:29.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70cdcaa700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70c8156a10 0x7f70c806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:29.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70cd4a9700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f70c815d390 0x7f70c81608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39068/0 (socket says 172.21.15.12:39068) 2023-07-08T12:39:29.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70cd4a9700 1 -- 172.21.15.12:0/1737984922 learned_addr learned my addr 172.21.15.12:0/1737984922 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:29.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70ce4ab700 1 --2- 172.21.15.12:0/1737984922 >> v2:172.21.15.160:3300/0 conn(0x7f70c80cc240 0x7f70c80cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:29.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70cdcaa700 1 -- 172.21.15.12:0/1737984922 >> v2:172.21.15.160:3300/0 conn(0x7f70c80cc240 msgr2=0x7f70c80cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:29.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70cdcaa700 1 --2- 172.21.15.12:0/1737984922 >> v2:172.21.15.160:3300/0 conn(0x7f70c80cc240 0x7f70c80cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70cdcaa700 1 -- 172.21.15.12:0/1737984922 >> v1:172.21.15.12:6789/0 conn(0x7f70c815d390 legacy=0x7f70c81608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:29.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70cdcaa700 1 -- 172.21.15.12:0/1737984922 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c806ec70 con 0x7f70c8156a10 2023-07-08T12:39:29.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70ccca8700 1 -- 172.21.15.12:0/1737984922 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131028907 0 0) 0x7f70c80ccbe0 con 0x7f70c815d390 2023-07-08T12:39:29.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70ce4ab700 1 --2- 172.21.15.12:0/1737984922 >> v2:172.21.15.160:3300/0 conn(0x7f70c80cc240 0x7f70c80cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:29.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70cdcaa700 1 --2- 172.21.15.12:0/1737984922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c8156a10 0x7f70c806e0c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f70c806c880 tx=0x7f70c4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165415037b0f3611 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:29.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70ccca8700 1 -- 172.21.15.12:0/1737984922 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4005e30 con 0x7f70c8156a10 2023-07-08T12:39:29.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70ccca8700 1 -- 172.21.15.12:0/1737984922 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c4006030 con 0x7f70c8156a10 2023-07-08T12:39:29.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.939+0000 7f70ccca8700 1 -- 172.21.15.12:0/1737984922 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c40063c0 con 0x7f70c8156a10 2023-07-08T12:39:29.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.940+0000 7f70cff0e700 1 -- 172.21.15.12:0/1737984922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c8156a10 msgr2=0x7f70c806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:29.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.940+0000 7f70cff0e700 1 --2- 172.21.15.12:0/1737984922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c8156a10 0x7f70c806e0c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f70c806c880 tx=0x7f70c4009b00 comp rx=0 tx=0).stop 2023-07-08T12:39:29.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.940+0000 7f70cff0e700 1 -- 172.21.15.12:0/1737984922 shutdown_connections 2023-07-08T12:39:29.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.940+0000 7f70cff0e700 1 --2- 172.21.15.12:0/1737984922 >> v2:172.21.15.160:3300/0 conn(0x7f70c80cc240 0x7f70c80cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.940+0000 7f70cff0e700 1 --2- 172.21.15.12:0/1737984922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c8156a10 0x7f70c806e0c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.940+0000 7f70cff0e700 1 -- 172.21.15.12:0/1737984922 >> 172.21.15.12:0/1737984922 conn(0x7f70c8098b50 msgr2=0x7f70c806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:29.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.940+0000 7f70cff0e700 1 -- 172.21.15.12:0/1737984922 shutdown_connections 2023-07-08T12:39:29.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.940+0000 7f70cff0e700 1 -- 172.21.15.12:0/1737984922 wait complete. 2023-07-08T12:39:29.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.941+0000 7f70cff0e700 1 Processor -- start 2023-07-08T12:39:29.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.941+0000 7f70cff0e700 1 -- start start 2023-07-08T12:39:29.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cff0e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70c80cc240 0x7f70c80b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:29.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cff0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c8156a10 0x7f70c80ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:29.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cff0e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70c815d390 0x7f70c815f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:29.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cff0e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c81502c0 con 0x7f70c8156a10 2023-07-08T12:39:29.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cff0e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c815fdf0 con 0x7f70c815d390 2023-07-08T12:39:29.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cff0e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c80b2790 con 0x7f70c80cc240 2023-07-08T12:39:29.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cd4a9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c8156a10 0x7f70c80ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:29.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cd4a9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c8156a10 0x7f70c80ad810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51680/0 (socket says 172.21.15.12:51680) 2023-07-08T12:39:29.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cd4a9700 1 -- 172.21.15.12:0/3010485323 learned_addr learned my addr 172.21.15.12:0/3010485323 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:29.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70ce4ab700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70c815d390 0x7f70c815f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:29.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cdcaa700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70c80cc240 0x7f70c80b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:29.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cd4a9700 1 -- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70c80cc240 msgr2=0x7f70c80b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:29.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cd4a9700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70c80cc240 0x7f70c80b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cd4a9700 1 -- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70c815d390 msgr2=0x7f70c815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:29.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cd4a9700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70c815d390 0x7f70c815f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:29.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cd4a9700 1 -- 172.21.15.12:0/3010485323 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c4009430 con 0x7f70c8156a10 2023-07-08T12:39:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cdcaa700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70c80cc240 0x7f70c80b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.942+0000 7f70cd4a9700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c8156a10 0x7f70c80ad810 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f70b80011e0 tx=0x7f70b800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.943+0000 7f70be7fc700 1 -- 172.21.15.12:0/3010485323 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b8004c80 con 0x7f70c8156a10 2023-07-08T12:39:29.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.943+0000 7f70be7fc700 1 -- 172.21.15.12:0/3010485323 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b8004e00 con 0x7f70c8156a10 2023-07-08T12:39:29.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.943+0000 7f70cff0e700 1 -- 172.21.15.12:0/3010485323 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c80b2a70 con 0x7f70c8156a10 2023-07-08T12:39:29.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.943+0000 7f70be7fc700 1 -- 172.21.15.12:0/3010485323 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b8007310 con 0x7f70c8156a10 2023-07-08T12:39:29.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.943+0000 7f70cff0e700 1 -- 172.21.15.12:0/3010485323 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c80b2e00 con 0x7f70c8156a10 2023-07-08T12:39:29.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.944+0000 7f70cff0e700 1 -- 172.21.15.12:0/3010485323 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c80b3b50 con 0x7f70c8156a10 2023-07-08T12:39:29.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.944+0000 7f70be7fc700 1 -- 172.21.15.12:0/3010485323 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70b8007470 con 0x7f70c8156a10 2023-07-08T12:39:29.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.945+0000 7f70be7fc700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f70ac06ace0 0x7f70ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:29.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.945+0000 7f70be7fc700 1 -- 172.21.15.12:0/3010485323 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f70b80896c0 con 0x7f70c8156a10 2023-07-08T12:39:29.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.945+0000 7f70cdcaa700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f70ac06ace0 0x7f70ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:29.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.945+0000 7f70cdcaa700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f70ac06ace0 0x7f70ac06d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f70c4013040 tx=0x7f70c4008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:29.949+0000 7f70be7fc700 1 -- 172.21.15.12:0/3010485323 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f70b8054980 con 0x7f70c8156a10 2023-07-08T12:39:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.144+0000 7f70cff0e700 1 -- 172.21.15.12:0/3010485323 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f70c80b3210 con 0x7f70c8156a10 2023-07-08T12:39:30.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.790+0000 7f70be7fc700 1 -- 172.21.15.12:0/3010485323 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v40) v1 ==== 170+0+0 (secure 0 0 0) 0x7f70b8057fa0 con 0x7f70c8156a10 2023-07-08T12:39:30.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.795+0000 7f70cff0e700 1 -- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f70ac06ace0 msgr2=0x7f70ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:30.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.795+0000 7f70cff0e700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f70ac06ace0 0x7f70ac06d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f70c4013040 tx=0x7f70c4008070 comp rx=0 tx=0).stop 2023-07-08T12:39:30.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.795+0000 7f70cff0e700 1 -- 172.21.15.12:0/3010485323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c8156a10 msgr2=0x7f70c80ad810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:30.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.795+0000 7f70cff0e700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c8156a10 0x7f70c80ad810 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f70b80011e0 tx=0x7f70b800cae0 comp rx=0 tx=0).stop 2023-07-08T12:39:30.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.795+0000 7f70cff0e700 1 -- 172.21.15.12:0/3010485323 shutdown_connections 2023-07-08T12:39:30.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.795+0000 7f70cff0e700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f70ac06ace0 0x7f70ac06d190 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f70c4013040 tx=0x7f70c4008070 comp rx=0 tx=0).stop 2023-07-08T12:39:30.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.795+0000 7f70cff0e700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70c815d390 0x7f70c815f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:30.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.795+0000 7f70cff0e700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70c80cc240 0x7f70c80b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:30.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.795+0000 7f70cff0e700 1 --2- 172.21.15.12:0/3010485323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c8156a10 0x7f70c80ad810 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:30.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.795+0000 7f70cff0e700 1 -- 172.21.15.12:0/3010485323 >> 172.21.15.12:0/3010485323 conn(0x7f70c8098b50 msgr2=0x7f70c81608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:30.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.801+0000 7f70cff0e700 1 -- 172.21.15.12:0/3010485323 shutdown_connections 2023-07-08T12:39:30.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.801+0000 7f70cff0e700 1 -- 172.21.15.12:0/3010485323 wait complete. 2023-07-08T12:39:30.807 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' created 2023-07-08T12:39:30.824 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-08T12:39:30.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.956+0000 7fac7d1a4700 1 Processor -- start 2023-07-08T12:39:30.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.956+0000 7fac7d1a4700 1 -- start start 2023-07-08T12:39:30.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac7d1a4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac780b12b0 0x7fac780ac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:30.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac7d1a4700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7806d2b0 con 0x7fac7815a190 2023-07-08T12:39:30.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac7d1a4700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7806d590 con 0x7fac780ac960 2023-07-08T12:39:30.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac7d1a4700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac7815a560 con 0x7fac780b12b0 2023-07-08T12:39:30.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac7659c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac780b12b0 0x7fac780ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:30.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac76d9d700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fac7815a190 0x7fac7806e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39082/0 (socket says 172.21.15.12:39082) 2023-07-08T12:39:30.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac76d9d700 1 -- 172.21.15.12:0/2958285205 learned_addr learned my addr 172.21.15.12:0/2958285205 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:30.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac7659c700 1 -- 172.21.15.12:0/2958285205 >> v1:172.21.15.160:6789/0 conn(0x7fac780ac960 legacy=0x7fac78163c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:30.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac7659c700 1 -- 172.21.15.12:0/2958285205 >> v1:172.21.15.12:6789/0 conn(0x7fac7815a190 legacy=0x7fac7806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:30.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac7659c700 1 -- 172.21.15.12:0/2958285205 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7806d870 con 0x7fac780b12b0 2023-07-08T12:39:30.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac75d9b700 1 -- 172.21.15.12:0/2958285205 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405258504 0 0) 0x7fac7806d2b0 con 0x7fac7815a190 2023-07-08T12:39:30.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac7659c700 1 --2- 172.21.15.12:0/2958285205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac780b12b0 0x7fac780ac420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009840 tx=0x7fac6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e471f69b5a265644 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:30.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac75d9b700 1 -- 172.21.15.12:0/2958285205 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c004230 con 0x7fac780b12b0 2023-07-08T12:39:30.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac75d9b700 1 -- 172.21.15.12:0/2958285205 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c004390 con 0x7fac780b12b0 2023-07-08T12:39:30.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.957+0000 7fac75d9b700 1 -- 172.21.15.12:0/2958285205 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0036a0 con 0x7fac780b12b0 2023-07-08T12:39:30.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.958+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2958285205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac780b12b0 msgr2=0x7fac780ac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:30.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.958+0000 7fac7d1a4700 1 --2- 172.21.15.12:0/2958285205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac780b12b0 0x7fac780ac420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009840 tx=0x7fac6c009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:30.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.958+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2958285205 shutdown_connections 2023-07-08T12:39:30.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.958+0000 7fac7d1a4700 1 --2- 172.21.15.12:0/2958285205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac780b12b0 0x7fac780ac420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:30.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.958+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2958285205 >> 172.21.15.12:0/2958285205 conn(0x7fac78098b50 msgr2=0x7fac7806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:30.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.958+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2958285205 shutdown_connections 2023-07-08T12:39:30.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.958+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2958285205 wait complete. 2023-07-08T12:39:30.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.959+0000 7fac7d1a4700 1 Processor -- start 2023-07-08T12:39:30.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.959+0000 7fac7d1a4700 1 -- start start 2023-07-08T12:39:30.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.960+0000 7fac7d1a4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac780ac960 0x7fac780c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:30.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.960+0000 7fac7d1a4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac780b12b0 0x7fac78153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:30.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.960+0000 7fac7d1a4700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac7815a190 0x7fac781f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:30.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.960+0000 7fac7d1a4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac781550c0 con 0x7fac780b12b0 2023-07-08T12:39:30.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.960+0000 7fac7d1a4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac781f3210 con 0x7fac780ac960 2023-07-08T12:39:30.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.960+0000 7fac7d1a4700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac781f2f90 con 0x7fac7815a190 2023-07-08T12:39:30.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.960+0000 7fac7659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac780b12b0 0x7fac78153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:30.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.960+0000 7fac7659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac780b12b0 0x7fac78153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51694/0 (socket says 172.21.15.12:51694) 2023-07-08T12:39:30.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.960+0000 7fac7659c700 1 -- 172.21.15.12:0/2436846530 learned_addr learned my addr 172.21.15.12:0/2436846530 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:30.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.960+0000 7fac76d9d700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac780ac960 0x7fac780c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:30.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.961+0000 7fac7759e700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac7815a190 0x7fac781f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:30.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.961+0000 7fac7659c700 1 -- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac7815a190 msgr2=0x7fac781f2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:30.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.961+0000 7fac7659c700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac7815a190 0x7fac781f2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:30.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.961+0000 7fac7659c700 1 -- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac780ac960 msgr2=0x7fac780c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:30.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.961+0000 7fac7659c700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac780ac960 0x7fac780c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:30.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.961+0000 7fac7659c700 1 -- 172.21.15.12:0/2436846530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c0094a0 con 0x7fac780b12b0 2023-07-08T12:39:30.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.961+0000 7fac7759e700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac7815a190 0x7fac781f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:30.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.961+0000 7fac7659c700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac780b12b0 0x7fac78153e20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fac78071290 tx=0x7fac6c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:30.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.961+0000 7fac5f7fe700 1 -- 172.21.15.12:0/2436846530 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c011630 con 0x7fac780b12b0 2023-07-08T12:39:30.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.961+0000 7fac5f7fe700 1 -- 172.21.15.12:0/2436846530 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c011790 con 0x7fac780b12b0 2023-07-08T12:39:30.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.961+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2436846530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac78154720 con 0x7fac780b12b0 2023-07-08T12:39:30.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.961+0000 7fac5f7fe700 1 -- 172.21.15.12:0/2436846530 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c011b40 con 0x7fac780b12b0 2023-07-08T12:39:30.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.961+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2436846530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac78154ce0 con 0x7fac780b12b0 2023-07-08T12:39:30.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.962+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2436846530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac78155200 con 0x7fac780b12b0 2023-07-08T12:39:30.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.963+0000 7fac5f7fe700 1 -- 172.21.15.12:0/2436846530 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fac6c011ca0 con 0x7fac780b12b0 2023-07-08T12:39:30.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.963+0000 7fac5f7fe700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac6406ace0 0x7fac6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:30.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.963+0000 7fac5f7fe700 1 -- 172.21.15.12:0/2436846530 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fac6c08add0 con 0x7fac780b12b0 2023-07-08T12:39:30.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.963+0000 7fac76d9d700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac6406ace0 0x7fac6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:30.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.964+0000 7fac76d9d700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac6406ace0 0x7fac6406d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fac68007fd0 tx=0x7fac68009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:30.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:30.968+0000 7fac5f7fe700 1 -- 172.21.15.12:0/2436846530 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fac78155200 con 0x7fac780b12b0 2023-07-08T12:39:31.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.211+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2436846530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fac78155200 con 0x7fac780b12b0 2023-07-08T12:39:31.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.819+0000 7fac5f7fe700 1 -- 172.21.15.12:0/2436846530 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fac78155200 con 0x7fac780b12b0 2023-07-08T12:39:31.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.824+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac6406ace0 msgr2=0x7fac6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:31.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.824+0000 7fac7d1a4700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac6406ace0 0x7fac6406d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fac68007fd0 tx=0x7fac68009040 comp rx=0 tx=0).stop 2023-07-08T12:39:31.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.824+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2436846530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac780b12b0 msgr2=0x7fac78153e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:31.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.824+0000 7fac7d1a4700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac780b12b0 0x7fac78153e20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fac78071290 tx=0x7fac6c0047c0 comp rx=0 tx=0).stop 2023-07-08T12:39:31.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.825+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2436846530 shutdown_connections 2023-07-08T12:39:31.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.825+0000 7fac7d1a4700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fac6406ace0 0x7fac6406d190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:31.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.825+0000 7fac7d1a4700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac7815a190 0x7fac781f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:31.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.825+0000 7fac7d1a4700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac780ac960 0x7fac780c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:31.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.825+0000 7fac7d1a4700 1 --2- 172.21.15.12:0/2436846530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac780b12b0 0x7fac78153e20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:31.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.825+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2436846530 >> 172.21.15.12:0/2436846530 conn(0x7fac78098b50 msgr2=0x7fac780af670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:31.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.825+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2436846530 shutdown_connections 2023-07-08T12:39:31.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.825+0000 7fac7d1a4700 1 -- 172.21.15.12:0/2436846530 wait complete. 2023-07-08T12:39:31.847 INFO:teuthology.orchestra.run.smithi012.stderr:new fs with metadata pool 6 and data pool 7 2023-07-08T12:39:31.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:39:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.986+0000 7f64f2b74700 1 Processor -- start 2023-07-08T12:39:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.986+0000 7f64f2b74700 1 -- start start 2023-07-08T12:39:31.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64f2b74700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f64ec159c30 0x7f64ec0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:31.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64f2b74700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f64ec160790 0x7f64ec06ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:31.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64f2b74700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec15c890 con 0x7f64ec069f30 2023-07-08T12:39:31.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64f2b74700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec15c730 con 0x7f64ec159c30 2023-07-08T12:39:31.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64f2b74700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec0cf590 con 0x7f64ec160790 2023-07-08T12:39:31.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64f0910700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f64ec159c30 0x7f64ec0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:31.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64f0910700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f64ec159c30 0x7f64ec0cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51714/0 (socket says 172.21.15.12:51714) 2023-07-08T12:39:31.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64f0910700 1 -- 172.21.15.12:0/1163782797 learned_addr learned my addr 172.21.15.12:0/1163782797 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:31.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64ebfff700 1 --2- 172.21.15.12:0/1163782797 >> v2:172.21.15.160:3300/0 conn(0x7f64ec160790 0x7f64ec06ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:31.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64f0910700 1 -- 172.21.15.12:0/1163782797 >> v2:172.21.15.160:3300/0 conn(0x7f64ec160790 msgr2=0x7f64ec06ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:31.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64f0910700 1 --2- 172.21.15.12:0/1163782797 >> v2:172.21.15.160:3300/0 conn(0x7f64ec160790 0x7f64ec06ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:31.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64f0910700 1 -- 172.21.15.12:0/1163782797 >> v1:172.21.15.160:6789/0 conn(0x7f64ec069f30 legacy=0x7f64ec15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:31.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64f0910700 1 -- 172.21.15.12:0/1163782797 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec160c90 con 0x7f64ec159c30 2023-07-08T12:39:31.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64ebfff700 1 --2- 172.21.15.12:0/1163782797 >> v2:172.21.15.160:3300/0 conn(0x7f64ec160790 0x7f64ec06ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:31.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64f0910700 1 --2- 172.21.15.12:0/1163782797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64ec159c30 0x7f64ec0cf050 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0b2370 tx=0x7f64e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1985955ebc0d680 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:31.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64eb7fe700 1 -- 172.21.15.12:0/1163782797 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e000b860 con 0x7f64ec159c30 2023-07-08T12:39:31.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64eb7fe700 1 -- 172.21.15.12:0/1163782797 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e000b9c0 con 0x7f64ec159c30 2023-07-08T12:39:31.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.987+0000 7f64eb7fe700 1 -- 172.21.15.12:0/1163782797 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e000bd50 con 0x7f64ec159c30 2023-07-08T12:39:31.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.988+0000 7f64f2b74700 1 -- 172.21.15.12:0/1163782797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64ec159c30 msgr2=0x7f64ec0cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:31.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.988+0000 7f64f2b74700 1 --2- 172.21.15.12:0/1163782797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64ec159c30 0x7f64ec0cf050 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0b2370 tx=0x7f64e0009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:31.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.988+0000 7f64f2b74700 1 -- 172.21.15.12:0/1163782797 shutdown_connections 2023-07-08T12:39:31.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.988+0000 7f64f2b74700 1 --2- 172.21.15.12:0/1163782797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64ec159c30 0x7f64ec0cf050 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:31.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.988+0000 7f64f2b74700 1 --2- 172.21.15.12:0/1163782797 >> v2:172.21.15.160:3300/0 conn(0x7f64ec160790 0x7f64ec06ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:31.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.988+0000 7f64f2b74700 1 -- 172.21.15.12:0/1163782797 >> 172.21.15.12:0/1163782797 conn(0x7f64ec0cc290 msgr2=0x7f64ec0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:31.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.988+0000 7f64f2b74700 1 -- 172.21.15.12:0/1163782797 shutdown_connections 2023-07-08T12:39:31.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.988+0000 7f64f2b74700 1 -- 172.21.15.12:0/1163782797 wait complete. 2023-07-08T12:39:31.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.989+0000 7f64f2b74700 1 Processor -- start 2023-07-08T12:39:31.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.989+0000 7f64f2b74700 1 -- start start 2023-07-08T12:39:31.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.990+0000 7f64f2b74700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec069f30 0x7f64ec0b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:31.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.990+0000 7f64f2b74700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64ec159c30 0x7f64ec0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:31.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.990+0000 7f64f2b74700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64ec160790 0x7f64ec150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:31.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.990+0000 7f64f2b74700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0b3520 con 0x7f64ec159c30 2023-07-08T12:39:31.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.990+0000 7f64f2b74700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0b2b50 con 0x7f64ec160790 2023-07-08T12:39:31.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.990+0000 7f64f2b74700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec150e40 con 0x7f64ec069f30 2023-07-08T12:39:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.990+0000 7f64ebfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64ec159c30 0x7f64ec0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.990+0000 7f64ebfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64ec159c30 0x7f64ec0b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51718/0 (socket says 172.21.15.12:51718) 2023-07-08T12:39:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.990+0000 7f64ebfff700 1 -- 172.21.15.12:0/2202042592 learned_addr learned my addr 172.21.15.12:0/2202042592 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.990+0000 7f64f0910700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec069f30 0x7f64ec0b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.990+0000 7f64f1111700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64ec160790 0x7f64ec150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.991+0000 7f64ebfff700 1 -- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec069f30 msgr2=0x7f64ec0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.991+0000 7f64ebfff700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec069f30 0x7f64ec0b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.991+0000 7f64ebfff700 1 -- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64ec160790 msgr2=0x7f64ec150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:31.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.991+0000 7f64ebfff700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64ec160790 0x7f64ec150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:31.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.991+0000 7f64ebfff700 1 -- 172.21.15.12:0/2202042592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e0009550 con 0x7f64ec159c30 2023-07-08T12:39:31.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.991+0000 7f64ebfff700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64ec159c30 0x7f64ec0b2fe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f64d800e810 tx=0x7f64d800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:31.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.991+0000 7f64e97fa700 1 -- 172.21.15.12:0/2202042592 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d8013610 con 0x7f64ec159c30 2023-07-08T12:39:31.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.991+0000 7f64e97fa700 1 -- 172.21.15.12:0/2202042592 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d8013790 con 0x7f64ec159c30 2023-07-08T12:39:31.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.991+0000 7f64f2b74700 1 -- 172.21.15.12:0/2202042592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec0b3cd0 con 0x7f64ec159c30 2023-07-08T12:39:31.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.991+0000 7f64e97fa700 1 -- 172.21.15.12:0/2202042592 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d8013b40 con 0x7f64ec159c30 2023-07-08T12:39:31.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.991+0000 7f64f2b74700 1 -- 172.21.15.12:0/2202042592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec0b27e0 con 0x7f64ec159c30 2023-07-08T12:39:31.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.992+0000 7f64f2b74700 1 -- 172.21.15.12:0/2202042592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec0b41f0 con 0x7f64ec159c30 2023-07-08T12:39:31.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.992+0000 7f64e97fa700 1 -- 172.21.15.12:0/2202042592 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f64d8013ca0 con 0x7f64ec159c30 2023-07-08T12:39:31.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.993+0000 7f64e97fa700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f64d006ad10 0x7f64d006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.993+0000 7f64e97fa700 1 -- 172.21.15.12:0/2202042592 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f64d8089130 con 0x7f64ec159c30 2023-07-08T12:39:32.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.993+0000 7f64f0910700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f64d006ad10 0x7f64d006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.994+0000 7f64f0910700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f64d006ad10 0x7f64d006d1c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f64e0000c00 tx=0x7f64e0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:32.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:31.997+0000 7f64e97fa700 1 -- 172.21.15.12:0/2202042592 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f64ec0b41f0 con 0x7f64ec159c30 2023-07-08T12:39:32.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.197+0000 7f64f2b74700 1 -- 172.21.15.12:0/2202042592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64ec0b2640 con 0x7f64ec159c30 2023-07-08T12:39:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.198+0000 7f64e97fa700 1 -- 172.21.15.12:0/2202042592 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18501 (secure 0 0 0) 0x7f64ec0b2640 con 0x7f64ec159c30 2023-07-08T12:39:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:32.201 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":41,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:39:31.212871+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:39:29.579249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-08T12:39:30.175950+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:39:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.200+0000 7f64f2b74700 1 -- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f64d006ad10 msgr2=0x7f64d006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.200+0000 7f64f2b74700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f64d006ad10 0x7f64d006d1c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f64e0000c00 tx=0x7f64e0008030 comp rx=0 tx=0).stop 2023-07-08T12:39:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.200+0000 7f64f2b74700 1 -- 172.21.15.12:0/2202042592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64ec159c30 msgr2=0x7f64ec0b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.200+0000 7f64f2b74700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64ec159c30 0x7f64ec0b2fe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f64d800e810 tx=0x7f64d800eb20 comp rx=0 tx=0).stop 2023-07-08T12:39:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.200+0000 7f64f2b74700 1 -- 172.21.15.12:0/2202042592 shutdown_connections 2023-07-08T12:39:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.200+0000 7f64f2b74700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f64d006ad10 0x7f64d006d1c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.200+0000 7f64f2b74700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64ec160790 0x7f64ec150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.200+0000 7f64f2b74700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec069f30 0x7f64ec0b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.200+0000 7f64f2b74700 1 --2- 172.21.15.12:0/2202042592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64ec159c30 0x7f64ec0b2fe0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.200+0000 7f64f2b74700 1 -- 172.21.15.12:0/2202042592 >> 172.21.15.12:0/2202042592 conn(0x7f64ec0cc290 msgr2=0x7f64ec158f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:32.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.200+0000 7f64f2b74700 1 -- 172.21.15.12:0/2202042592 shutdown_connections 2023-07-08T12:39:32.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.200+0000 7f64f2b74700 1 -- 172.21.15.12:0/2202042592 wait complete. 2023-07-08T12:39:32.215 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:39:32.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.338+0000 7f909c05c700 1 Processor -- start 2023-07-08T12:39:32.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.338+0000 7f909c05c700 1 -- start start 2023-07-08T12:39:32.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f909c05c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90940d1cd0 0x7f90940cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f909c05c700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90940cf3b0 con 0x7f90940b4cc0 2023-07-08T12:39:32.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f909c05c700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90940b86f0 con 0x7f90940b8320 2023-07-08T12:39:32.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f909c05c700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90940cf0d0 con 0x7f90940d1cd0 2023-07-08T12:39:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f90995f7700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f90940b4cc0 0x7f90940d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39124/0 (socket says 172.21.15.12:39124) 2023-07-08T12:39:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f90995f7700 1 -- 172.21.15.12:0/4084162231 learned_addr learned my addr 172.21.15.12:0/4084162231 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f909a5f9700 1 --2- 172.21.15.12:0/4084162231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90940d1cd0 0x7f90940cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844414083 0 0) 0x7f90940cf3b0 con 0x7f90940b4cc0 2023-07-08T12:39:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90800035b0 con 0x7f90940b4cc0 2023-07-08T12:39:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466684425 0 0) 0x7f90940b86f0 con 0x7f90940b8320 2023-07-08T12:39:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90940cf3b0 con 0x7f90940b8320 2023-07-08T12:39:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3329398814 0 0) 0x7f90800035b0 con 0x7f90940b4cc0 2023-07-08T12:39:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90940d1cd0 msgr2=0x7f90940cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f9098df6700 1 --2- 172.21.15.12:0/4084162231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90940d1cd0 0x7f90940cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 >> v1:172.21.15.160:6789/0 conn(0x7f90940b8320 legacy=0x7f90940b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90940d22c0 con 0x7f90940b4cc0 2023-07-08T12:39:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.339+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f9084001670 con 0x7f90940b4cc0 2023-07-08T12:39:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 >> v1:172.21.15.12:6789/0 conn(0x7f90940b4cc0 legacy=0x7f90940d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f9098df6700 1 --2- 172.21.15.12:0/4084162231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90940d1cd0 0x7f9080003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f9098df6700 1 --2- 172.21.15.12:0/4084162231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9080004120 0x7f9080004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f9098df6700 1 --2- 172.21.15.12:0/4084162231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9080004d90 0x7f9080005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90940cf0d0 con 0x7f9080004d90 2023-07-08T12:39:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908000bc00 con 0x7f9080004120 2023-07-08T12:39:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908000bdc0 con 0x7f90940d1cd0 2023-07-08T12:39:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f909a5f9700 1 --2- 172.21.15.12:0/4084162231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9080004d90 0x7f9080005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f9099df8700 1 --2- 172.21.15.12:0/4084162231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90940d1cd0 0x7f9080003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f90995f7700 1 --2- 172.21.15.12:0/4084162231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9080004120 0x7f9080004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f909a5f9700 1 -- 172.21.15.12:0/4084162231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90940d1cd0 msgr2=0x7f9080003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f909a5f9700 1 --2- 172.21.15.12:0/4084162231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90940d1cd0 0x7f9080003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f909a5f9700 1 -- 172.21.15.12:0/4084162231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9080004120 msgr2=0x7f9080004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f909a5f9700 1 --2- 172.21.15.12:0/4084162231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9080004120 0x7f9080004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f909a5f9700 1 -- 172.21.15.12:0/4084162231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90800035b0 con 0x7f9080004d90 2023-07-08T12:39:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f909a5f9700 1 --2- 172.21.15.12:0/4084162231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9080004d90 0x7f9080005200 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f908c009f90 tx=0x7f9080005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264d4c3b205ae4a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c00ec30 con 0x7f9080004d90 2023-07-08T12:39:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908c00f040 con 0x7f9080004d90 2023-07-08T12:39:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.340+0000 7f9098df6700 1 -- 172.21.15.12:0/4084162231 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c014480 con 0x7f9080004d90 2023-07-08T12:39:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.341+0000 7f909c05c700 1 -- 172.21.15.12:0/4084162231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9080004d90 msgr2=0x7f9080005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.341+0000 7f909c05c700 1 --2- 172.21.15.12:0/4084162231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9080004d90 0x7f9080005200 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f908c009f90 tx=0x7f9080005880 comp rx=0 tx=0).stop 2023-07-08T12:39:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.341+0000 7f909a5f9700 1 -- 172.21.15.12:0/4084162231 reap_dead start 2023-07-08T12:39:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.341+0000 7f909c05c700 1 -- 172.21.15.12:0/4084162231 shutdown_connections 2023-07-08T12:39:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.341+0000 7f909c05c700 1 -- 172.21.15.12:0/4084162231 >> 172.21.15.12:0/4084162231 conn(0x7f90940722b0 msgr2=0x7f90940cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.341+0000 7f909c05c700 1 -- 172.21.15.12:0/4084162231 shutdown_connections 2023-07-08T12:39:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.341+0000 7f909c05c700 1 -- 172.21.15.12:0/4084162231 wait complete. 2023-07-08T12:39:32.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.343+0000 7f909c05c700 1 Processor -- start 2023-07-08T12:39:32.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.343+0000 7f909c05c700 1 -- start start 2023-07-08T12:39:32.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909c05c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9094153160 0x7f90940c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909c05c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90940b5dd0 0x7f90941e7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909c05c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90941e5990 0x7f90941e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909c05c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9094153530 con 0x7f90941e5990 2023-07-08T12:39:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909c05c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90941e6340 con 0x7f90940b5dd0 2023-07-08T12:39:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909c05c700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90940b62d0 con 0x7f9094153160 2023-07-08T12:39:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909a5f9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90941e5990 0x7f90941e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909a5f9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90941e5990 0x7f90941e5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51742/0 (socket says 172.21.15.12:51742) 2023-07-08T12:39:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909a5f9700 1 -- 172.21.15.12:0/2544177054 learned_addr learned my addr 172.21.15.12:0/2544177054 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f9099df8700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9094153160 0x7f90940c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f90995f7700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90940b5dd0 0x7f90941e7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909a5f9700 1 -- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9094153160 msgr2=0x7f90940c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:32.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909a5f9700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9094153160 0x7f90940c9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909a5f9700 1 -- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90940b5dd0 msgr2=0x7f90941e7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:32.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909a5f9700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90940b5dd0 0x7f90941e7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909a5f9700 1 -- 172.21.15.12:0/2544177054 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90940b65b0 con 0x7f90941e5990 2023-07-08T12:39:32.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f90995f7700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90940b5dd0 0x7f90941e7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:32.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909a5f9700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90941e5990 0x7f90941e5e00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f908c00e700 tx=0x7f908c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:32.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f908a7fc700 1 -- 172.21.15.12:0/2544177054 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c014d50 con 0x7f90941e5990 2023-07-08T12:39:32.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f908a7fc700 1 -- 172.21.15.12:0/2544177054 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908c009410 con 0x7f90941e5990 2023-07-08T12:39:32.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.344+0000 7f909c05c700 1 -- 172.21.15.12:0/2544177054 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90940b6890 con 0x7f90941e5990 2023-07-08T12:39:32.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.345+0000 7f908a7fc700 1 -- 172.21.15.12:0/2544177054 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c003c60 con 0x7f90941e5990 2023-07-08T12:39:32.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.345+0000 7f909c05c700 1 -- 172.21.15.12:0/2544177054 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90940b6e50 con 0x7f90941e5990 2023-07-08T12:39:32.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.345+0000 7f909c05c700 1 -- 172.21.15.12:0/2544177054 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909404ea60 con 0x7f90941e5990 2023-07-08T12:39:32.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.346+0000 7f908a7fc700 1 -- 172.21.15.12:0/2544177054 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f908c023030 con 0x7f90941e5990 2023-07-08T12:39:32.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.346+0000 7f908a7fc700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f907806ace0 0x7f907806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.346+0000 7f908a7fc700 1 -- 172.21.15.12:0/2544177054 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f908c08acf0 con 0x7f90941e5990 2023-07-08T12:39:32.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.347+0000 7f9099df8700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f907806ace0 0x7f907806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.347+0000 7f9099df8700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f907806ace0 0x7f907806d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f90900010e0 tx=0x7f9090009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:32.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.351+0000 7f908a7fc700 1 -- 172.21.15.12:0/2544177054 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f908c055cf0 con 0x7f90941e5990 2023-07-08T12:39:32.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.548+0000 7f909c05c700 1 -- 172.21.15.12:0/2544177054 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90940b41b0 con 0x7f90941e5990 2023-07-08T12:39:32.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.550+0000 7f908a7fc700 1 -- 172.21.15.12:0/2544177054 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18501 (secure 0 0 0) 0x7f908c059310 con 0x7f90941e5990 2023-07-08T12:39:32.552 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:32.552 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":41,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:39:31.212871+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:39:29.579249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-08T12:39:30.175950+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:39:32.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.552+0000 7f909c05c700 1 -- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f907806ace0 msgr2=0x7f907806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:32.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.552+0000 7f909c05c700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f907806ace0 0x7f907806d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f90900010e0 tx=0x7f9090009040 comp rx=0 tx=0).stop 2023-07-08T12:39:32.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.552+0000 7f909c05c700 1 -- 172.21.15.12:0/2544177054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90941e5990 msgr2=0x7f90941e5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.552+0000 7f909c05c700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90941e5990 0x7f90941e5e00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f908c00e700 tx=0x7f908c009750 comp rx=0 tx=0).stop 2023-07-08T12:39:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.552+0000 7f909c05c700 1 -- 172.21.15.12:0/2544177054 shutdown_connections 2023-07-08T12:39:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.552+0000 7f909c05c700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f907806ace0 0x7f907806d190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.552+0000 7f909c05c700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90941e5990 0x7f90941e5e00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.552+0000 7f909c05c700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9094153160 0x7f90940c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.552+0000 7f909c05c700 1 --2- 172.21.15.12:0/2544177054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90940b5dd0 0x7f90941e7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.552+0000 7f909c05c700 1 -- 172.21.15.12:0/2544177054 >> 172.21.15.12:0/2544177054 conn(0x7f9080004d90 msgr2=0x7f90941583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.552+0000 7f909c05c700 1 -- 172.21.15.12:0/2544177054 shutdown_connections 2023-07-08T12:39:32.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.552+0000 7f909c05c700 1 -- 172.21.15.12:0/2544177054 wait complete. 2023-07-08T12:39:32.566 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-08T12:39:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.693+0000 7fcab1f6b700 1 Processor -- start 2023-07-08T12:39:32.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.693+0000 7fcab1f6b700 1 -- start start 2023-07-08T12:39:32.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcab1f6b700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcaac15a6d0 0x7fcaac15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcab1f6b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcaac161ce0 0x7fcaac1640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcab1f6b700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaac158130 con 0x7fcaac0ce560 2023-07-08T12:39:32.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcab1f6b700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaac15b730 con 0x7fcaac15a6d0 2023-07-08T12:39:32.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcab1f6b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac15b450 con 0x7fcaac161ce0 2023-07-08T12:39:32.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaab7fe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fcaac0ce560 0x7fcaac0ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39142/0 (socket says 172.21.15.12:39142) 2023-07-08T12:39:32.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaab7fe700 1 -- 172.21.15.12:0/1033764715 learned_addr learned my addr 172.21.15.12:0/1033764715 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:32.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850015130 0 0) 0x7fcaac158130 con 0x7fcaac0ce560 2023-07-08T12:39:32.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaaaffd700 1 --2- 172.21.15.12:0/1033764715 >> v2:172.21.15.160:3300/0 conn(0x7fcaac15a6d0 0x7fcaac15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaabfff700 1 --2- 172.21.15.12:0/1033764715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcaac161ce0 0x7fcaac1640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca940035b0 con 0x7fcaac0ce560 2023-07-08T12:39:32.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 624446344 0 0) 0x7fca940035b0 con 0x7fcaac0ce560 2023-07-08T12:39:32.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcaac161ce0 msgr2=0x7fcaac1640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:32.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaaa7fc700 1 --2- 172.21.15.12:0/1033764715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcaac161ce0 0x7fcaac1640c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 >> v2:172.21.15.160:3300/0 conn(0x7fcaac15a6d0 msgr2=0x7fcaac15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaaa7fc700 1 --2- 172.21.15.12:0/1033764715 >> v2:172.21.15.160:3300/0 conn(0x7fcaac15a6d0 0x7fcaac15ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac06e850 con 0x7fcaac0ce560 2023-07-08T12:39:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7fca9c001670 con 0x7fcaac0ce560 2023-07-08T12:39:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.694+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 >> v1:172.21.15.12:6789/0 conn(0x7fcaac0ce560 legacy=0x7fcaac0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaaa7fc700 1 --2- 172.21.15.12:0/1033764715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcaac161ce0 0x7fca94003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaaa7fc700 1 --2- 172.21.15.12:0/1033764715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca94003f80 0x7fca94004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaaa7fc700 1 --2- 172.21.15.12:0/1033764715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca94004d90 0x7fca94005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac15b730 con 0x7fca94004d90 2023-07-08T12:39:32.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac15b450 con 0x7fca94003f80 2023-07-08T12:39:32.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9400fe20 con 0x7fcaac161ce0 2023-07-08T12:39:32.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaabfff700 1 --2- 172.21.15.12:0/1033764715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca94004d90 0x7fca94005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaab7fe700 1 --2- 172.21.15.12:0/1033764715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcaac161ce0 0x7fca94003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaaaffd700 1 --2- 172.21.15.12:0/1033764715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca94003f80 0x7fca94004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaabfff700 1 -- 172.21.15.12:0/1033764715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcaac161ce0 msgr2=0x7fca94003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaabfff700 1 --2- 172.21.15.12:0/1033764715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcaac161ce0 0x7fca94003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaabfff700 1 -- 172.21.15.12:0/1033764715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca94003f80 msgr2=0x7fca94004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaabfff700 1 --2- 172.21.15.12:0/1033764715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca94003f80 0x7fca94004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaabfff700 1 -- 172.21.15.12:0/1033764715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca940035b0 con 0x7fca94004d90 2023-07-08T12:39:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaabfff700 1 --2- 172.21.15.12:0/1033764715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca94004d90 0x7fca94005200 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcaac08a780 tx=0x7fca94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2de160bf9b4bca server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa400ebd0 con 0x7fca94004d90 2023-07-08T12:39:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa400f040 con 0x7fca94004d90 2023-07-08T12:39:32.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.695+0000 7fcaaa7fc700 1 -- 172.21.15.12:0/1033764715 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4014480 con 0x7fca94004d90 2023-07-08T12:39:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.696+0000 7fcab1f6b700 1 -- 172.21.15.12:0/1033764715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca94004d90 msgr2=0x7fca94005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.696+0000 7fcab1f6b700 1 --2- 172.21.15.12:0/1033764715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca94004d90 0x7fca94005200 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcaac08a780 tx=0x7fca94005880 comp rx=0 tx=0).stop 2023-07-08T12:39:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.696+0000 7fcaabfff700 1 -- 172.21.15.12:0/1033764715 reap_dead start 2023-07-08T12:39:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.696+0000 7fcab1f6b700 1 -- 172.21.15.12:0/1033764715 shutdown_connections 2023-07-08T12:39:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.696+0000 7fcab1f6b700 1 -- 172.21.15.12:0/1033764715 >> 172.21.15.12:0/1033764715 conn(0x7fcaac099c60 msgr2=0x7fcaac089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.696+0000 7fcab1f6b700 1 -- 172.21.15.12:0/1033764715 shutdown_connections 2023-07-08T12:39:32.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.696+0000 7fcab1f6b700 1 -- 172.21.15.12:0/1033764715 wait complete. 2023-07-08T12:39:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.697+0000 7fcab1f6b700 1 Processor -- start 2023-07-08T12:39:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.697+0000 7fcab1f6b700 1 -- start start 2023-07-08T12:39:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcab1f6b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcaac14ffd0 0x7fcaac153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcab1f6b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaac15d4a0 0x7fcaac153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcab1f6b700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcaac14e0d0 0x7fcaac15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcab1f6b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac14cf10 con 0x7fcaac15d4a0 2023-07-08T12:39:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcab1f6b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac14d1f0 con 0x7fcaac14ffd0 2023-07-08T12:39:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcab1f6b700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac14d4d0 con 0x7fcaac14e0d0 2023-07-08T12:39:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcaaaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaac15d4a0 0x7fcaac153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcaaaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaac15d4a0 0x7fcaac153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51760/0 (socket says 172.21.15.12:51760) 2023-07-08T12:39:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcaaaffd700 1 -- 172.21.15.12:0/899268166 learned_addr learned my addr 172.21.15.12:0/899268166 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcaab7fe700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcaac14ffd0 0x7fcaac153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcaabfff700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcaac14e0d0 0x7fcaac15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcaaaffd700 1 -- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcaac14e0d0 msgr2=0x7fcaac15b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcaaaffd700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcaac14e0d0 0x7fcaac15b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcaaaffd700 1 -- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcaac14ffd0 msgr2=0x7fcaac153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcaaaffd700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcaac14ffd0 0x7fcaac153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.698+0000 7fcaaaffd700 1 -- 172.21.15.12:0/899268166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac14d7b0 con 0x7fcaac15d4a0 2023-07-08T12:39:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.699+0000 7fcaaaffd700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaac15d4a0 0x7fcaac153790 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0011aa0 tx=0x7fcaa0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.699+0000 7fca93fff700 1 -- 172.21.15.12:0/899268166 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa000f9e0 con 0x7fcaac15d4a0 2023-07-08T12:39:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.699+0000 7fca93fff700 1 -- 172.21.15.12:0/899268166 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa000fb40 con 0x7fcaac15d4a0 2023-07-08T12:39:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.699+0000 7fca93fff700 1 -- 172.21.15.12:0/899268166 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0014440 con 0x7fcaac15d4a0 2023-07-08T12:39:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.699+0000 7fcab1f6b700 1 -- 172.21.15.12:0/899268166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaac14da90 con 0x7fcaac15d4a0 2023-07-08T12:39:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.699+0000 7fcab1f6b700 1 -- 172.21.15.12:0/899268166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaac1503a0 con 0x7fcaac15d4a0 2023-07-08T12:39:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.700+0000 7fcab1f6b700 1 -- 172.21.15.12:0/899268166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaac04ea60 con 0x7fcaac15d4a0 2023-07-08T12:39:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.700+0000 7fca93fff700 1 -- 172.21.15.12:0/899268166 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcaa00145a0 con 0x7fcaac15d4a0 2023-07-08T12:39:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.701+0000 7fca93fff700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fca9806ace0 0x7fca9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.701+0000 7fca93fff700 1 -- 172.21.15.12:0/899268166 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcaa0089e00 con 0x7fcaac15d4a0 2023-07-08T12:39:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.701+0000 7fcaab7fe700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fca9806ace0 0x7fca9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.702+0000 7fcaab7fe700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fca9806ace0 0x7fca9806d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fca9c000fb0 tx=0x7fca9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.705+0000 7fca93fff700 1 -- 172.21.15.12:0/899268166 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcaa0054d70 con 0x7fcaac15d4a0 2023-07-08T12:39:32.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:32.951+0000 7fcab1f6b700 1 -- 172.21.15.12:0/899268166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcaac15a210 con 0x7fcaac15d4a0 2023-07-08T12:39:33.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.836+0000 7fca93fff700 1 -- 172.21.15.12:0/899268166 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcaa0058390 con 0x7fcaac15d4a0 2023-07-08T12:39:33.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.838+0000 7fcab1f6b700 1 -- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fca9806ace0 msgr2=0x7fca9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:33.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.838+0000 7fcab1f6b700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fca9806ace0 0x7fca9806d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fca9c000fb0 tx=0x7fca9c009040 comp rx=0 tx=0).stop 2023-07-08T12:39:33.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.838+0000 7fcab1f6b700 1 -- 172.21.15.12:0/899268166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaac15d4a0 msgr2=0x7fcaac153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.838+0000 7fcab1f6b700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaac15d4a0 0x7fcaac153790 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0011aa0 tx=0x7fcaa0011e60 comp rx=0 tx=0).stop 2023-07-08T12:39:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.838+0000 7fcab1f6b700 1 -- 172.21.15.12:0/899268166 shutdown_connections 2023-07-08T12:39:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.838+0000 7fcab1f6b700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fca9806ace0 0x7fca9806d190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.838+0000 7fcab1f6b700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcaac14e0d0 0x7fcaac15b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.838+0000 7fcab1f6b700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcaac14ffd0 0x7fcaac153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.838+0000 7fcab1f6b700 1 --2- 172.21.15.12:0/899268166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaac15d4a0 0x7fcaac153790 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.839+0000 7fcab1f6b700 1 -- 172.21.15.12:0/899268166 >> 172.21.15.12:0/899268166 conn(0x7fcaac099c60 msgr2=0x7fcaac158a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.839+0000 7fcab1f6b700 1 -- 172.21.15.12:0/899268166 shutdown_connections 2023-07-08T12:39:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.839+0000 7fcab1f6b700 1 -- 172.21.15.12:0/899268166 wait complete. 2023-07-08T12:39:33.855 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-08T12:39:33.856 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-08T12:39:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.989+0000 7fedd7acc700 1 Processor -- start 2023-07-08T12:39:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.989+0000 7fedd7acc700 1 -- start start 2023-07-08T12:39:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.990+0000 7fedd7acc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fedd00cfc30 0x7fedd006b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.990+0000 7fedd7acc700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd00ce300 con 0x7fedd015fad0 2023-07-08T12:39:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.990+0000 7fedd7acc700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd015fea0 con 0x7fedd006f7f0 2023-07-08T12:39:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.990+0000 7fedd7acc700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd006e750 con 0x7fedd00cfc30 2023-07-08T12:39:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.990+0000 7fedd5067700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fedd00cfc30 0x7fedd006b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.990+0000 7fedd5868700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fedd015fad0 0x7fedd00cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39162/0 (socket says 172.21.15.12:39162) 2023-07-08T12:39:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.990+0000 7fedd5868700 1 -- 172.21.15.12:0/3415167272 learned_addr learned my addr 172.21.15.12:0/3415167272 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.990+0000 7fedd5067700 1 -- 172.21.15.12:0/3415167272 >> v1:172.21.15.160:6789/0 conn(0x7fedd006f7f0 legacy=0x7fedd00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.990+0000 7fedd5067700 1 -- 172.21.15.12:0/3415167272 >> v1:172.21.15.12:6789/0 conn(0x7fedd015fad0 legacy=0x7fedd00cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.990+0000 7fedd5067700 1 -- 172.21.15.12:0/3415167272 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd006ea30 con 0x7fedd00cfc30 2023-07-08T12:39:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.990+0000 7fedd4866700 1 -- 172.21.15.12:0/3415167272 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009381188 0 0) 0x7fedd00ce300 con 0x7fedd015fad0 2023-07-08T12:39:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.990+0000 7fedd5067700 1 --2- 172.21.15.12:0/3415167272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd00cfc30 0x7fedd006b040 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009840 tx=0x7fedcc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebec6aaa2cef9746 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.991+0000 7fedd4866700 1 -- 172.21.15.12:0/3415167272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc011830 con 0x7fedd00cfc30 2023-07-08T12:39:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.991+0000 7fedd4866700 1 -- 172.21.15.12:0/3415167272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedcc011990 con 0x7fedd00cfc30 2023-07-08T12:39:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.991+0000 7fedd4866700 1 -- 172.21.15.12:0/3415167272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc011d20 con 0x7fedd00cfc30 2023-07-08T12:39:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.991+0000 7fedd7acc700 1 -- 172.21.15.12:0/3415167272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd00cfc30 msgr2=0x7fedd006b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.991+0000 7fedd7acc700 1 --2- 172.21.15.12:0/3415167272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd00cfc30 0x7fedd006b040 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009840 tx=0x7fedcc009b50 comp rx=0 tx=0).stop 2023-07-08T12:39:33.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.991+0000 7fedd7acc700 1 -- 172.21.15.12:0/3415167272 shutdown_connections 2023-07-08T12:39:33.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.991+0000 7fedd7acc700 1 --2- 172.21.15.12:0/3415167272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd00cfc30 0x7fedd006b040 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:33.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.991+0000 7fedd7acc700 1 -- 172.21.15.12:0/3415167272 >> 172.21.15.12:0/3415167272 conn(0x7fedd0099c60 msgr2=0x7fedd0089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:33.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.991+0000 7fedd7acc700 1 -- 172.21.15.12:0/3415167272 shutdown_connections 2023-07-08T12:39:33.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.991+0000 7fedd7acc700 1 -- 172.21.15.12:0/3415167272 wait complete. 2023-07-08T12:39:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.992+0000 7fedd7acc700 1 Processor -- start 2023-07-08T12:39:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.993+0000 7fedd7acc700 1 -- start start 2023-07-08T12:39:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.993+0000 7fedd7acc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedd006f7f0 0x7fedd015aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.993+0000 7fedd7acc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd00cfc30 0x7fedd00b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.993+0000 7fedd7acc700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedd015fad0 0x7fedd015a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.993+0000 7fedd7acc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00c85a0 con 0x7fedd00cfc30 2023-07-08T12:39:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.993+0000 7fedd7acc700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00c8250 con 0x7fedd006f7f0 2023-07-08T12:39:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.993+0000 7fedd7acc700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00b2bf0 con 0x7fedd015fad0 2023-07-08T12:39:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.993+0000 7fedd5067700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd00cfc30 0x7fedd00b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.993+0000 7fedd5067700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd00cfc30 0x7fedd00b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51774/0 (socket says 172.21.15.12:51774) 2023-07-08T12:39:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.993+0000 7fedd5067700 1 -- 172.21.15.12:0/807154806 learned_addr learned my addr 172.21.15.12:0/807154806 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedd5868700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedd006f7f0 0x7fedd015aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedd6069700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedd015fad0 0x7fedd015a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedd5067700 1 -- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedd015fad0 msgr2=0x7fedd015a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedd5067700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedd015fad0 0x7fedd015a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedd5067700 1 -- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedd006f7f0 msgr2=0x7fedd015aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedd5067700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedd006f7f0 0x7fedd015aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedd5067700 1 -- 172.21.15.12:0/807154806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0094a0 con 0x7fedd00cfc30 2023-07-08T12:39:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedd6069700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedd015fad0 0x7fedd015a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedd5067700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd00cfc30 0x7fedd00b25a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0090b0 tx=0x7fedcc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedc67fc700 1 -- 172.21.15.12:0/807154806 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc01a8c0 con 0x7fedd00cfc30 2023-07-08T12:39:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedc67fc700 1 -- 172.21.15.12:0/807154806 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedcc01aa20 con 0x7fedd00cfc30 2023-07-08T12:39:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedc67fc700 1 -- 172.21.15.12:0/807154806 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc01ad90 con 0x7fedd00cfc30 2023-07-08T12:39:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedd7acc700 1 -- 172.21.15.12:0/807154806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd00b2ed0 con 0x7fedd00cfc30 2023-07-08T12:39:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.994+0000 7fedd7acc700 1 -- 172.21.15.12:0/807154806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd015b1e0 con 0x7fedd00cfc30 2023-07-08T12:39:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.995+0000 7fedd7acc700 1 -- 172.21.15.12:0/807154806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd00b33f0 con 0x7fedd00cfc30 2023-07-08T12:39:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.995+0000 7fedc67fc700 1 -- 172.21.15.12:0/807154806 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fedcc022070 con 0x7fedd00cfc30 2023-07-08T12:39:34.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.996+0000 7fedc67fc700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fedb406ace0 0x7fedb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:34.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.996+0000 7fedc67fc700 1 -- 172.21.15.12:0/807154806 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fedcc08a700 con 0x7fedd00cfc30 2023-07-08T12:39:34.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.996+0000 7fedd5868700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fedb406ace0 0x7fedb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:34.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:33.997+0000 7fedd5868700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fedb406ace0 0x7fedb406d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fedc0000f60 tx=0x7fedc0007db0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:34.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.000+0000 7fedc67fc700 1 -- 172.21.15.12:0/807154806 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fedcc0557f0 con 0x7fedd00cfc30 2023-07-08T12:39:34.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.246+0000 7fedd7acc700 1 -- 172.21.15.12:0/807154806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fedd004ea60 con 0x7fedd00cfc30 2023-07-08T12:39:34.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.838+0000 7fedc67fc700 1 -- 172.21.15.12:0/807154806 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fedcc058dd0 con 0x7fedd00cfc30 2023-07-08T12:39:34.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.841+0000 7fedd7acc700 1 -- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fedb406ace0 msgr2=0x7fedb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:34.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.841+0000 7fedd7acc700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fedb406ace0 0x7fedb406d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fedc0000f60 tx=0x7fedc0007db0 comp rx=0 tx=0).stop 2023-07-08T12:39:34.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.841+0000 7fedd7acc700 1 -- 172.21.15.12:0/807154806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd00cfc30 msgr2=0x7fedd00b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:34.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.841+0000 7fedd7acc700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd00cfc30 0x7fedd00b25a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0090b0 tx=0x7fedcc004750 comp rx=0 tx=0).stop 2023-07-08T12:39:34.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.841+0000 7fedd7acc700 1 -- 172.21.15.12:0/807154806 shutdown_connections 2023-07-08T12:39:34.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.841+0000 7fedd7acc700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fedb406ace0 0x7fedb406d190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:34.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.841+0000 7fedd7acc700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedd015fad0 0x7fedd015a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:34.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.841+0000 7fedd7acc700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedd006f7f0 0x7fedd015aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:34.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.841+0000 7fedd7acc700 1 --2- 172.21.15.12:0/807154806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd00cfc30 0x7fedd00b25a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:34.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.841+0000 7fedd7acc700 1 -- 172.21.15.12:0/807154806 >> 172.21.15.12:0/807154806 conn(0x7fedd0099c60 msgr2=0x7fedd0089fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:34.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.841+0000 7fedd7acc700 1 -- 172.21.15.12:0/807154806 shutdown_connections 2023-07-08T12:39:34.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.841+0000 7fedd7acc700 1 -- 172.21.15.12:0/807154806 wait complete. 2023-07-08T12:39:34.858 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-08T12:39:34.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-08T12:39:34.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.988+0000 7fcd792af700 1 Processor -- start 2023-07-08T12:39:34.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.988+0000 7fcd792af700 1 -- start start 2023-07-08T12:39:34.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.988+0000 7fcd792af700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd7415c1e0 0x7fcd740caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:34.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.988+0000 7fcd792af700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcd740cb470 0x7fcd7406ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:34.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.988+0000 7fcd792af700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd740ce6e0 con 0x7fcd7406bf90 2023-07-08T12:39:34.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.988+0000 7fcd792af700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd740d06d0 con 0x7fcd740cb470 2023-07-08T12:39:34.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.988+0000 7fcd792af700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740ceb70 con 0x7fcd7415c1e0 2023-07-08T12:39:34.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.989+0000 7fcd727fc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcd740cb470 0x7fcd7406ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:34.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.989+0000 7fcd727fc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcd740cb470 0x7fcd7406ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51780/0 (socket says 172.21.15.12:51780) 2023-07-08T12:39:34.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.989+0000 7fcd727fc700 1 -- 172.21.15.12:0/3627926919 learned_addr learned my addr 172.21.15.12:0/3627926919 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:34.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.989+0000 7fcd72ffd700 1 --2- 172.21.15.12:0/3627926919 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd7415c1e0 0x7fcd740caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:34.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.989+0000 7fcd727fc700 1 -- 172.21.15.12:0/3627926919 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd7415c1e0 msgr2=0x7fcd740caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:34.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.989+0000 7fcd727fc700 1 --2- 172.21.15.12:0/3627926919 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd7415c1e0 0x7fcd740caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:34.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.989+0000 7fcd727fc700 1 -- 172.21.15.12:0/3627926919 >> v1:172.21.15.160:6789/0 conn(0x7fcd7406bf90 legacy=0x7fcd7415fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:34.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.989+0000 7fcd727fc700 1 -- 172.21.15.12:0/3627926919 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd74070b60 con 0x7fcd740cb470 2023-07-08T12:39:34.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.989+0000 7fcd727fc700 1 --2- 172.21.15.12:0/3627926919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd740cb470 0x7fcd7406ba50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcd680098f0 tx=0x7fcd68009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491c6371a91924e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:34.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.989+0000 7fcd71ffb700 1 -- 172.21.15.12:0/3627926919 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800b8a0 con 0x7fcd740cb470 2023-07-08T12:39:34.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.989+0000 7fcd71ffb700 1 -- 172.21.15.12:0/3627926919 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6800ba00 con 0x7fcd740cb470 2023-07-08T12:39:34.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.989+0000 7fcd71ffb700 1 -- 172.21.15.12:0/3627926919 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800bd90 con 0x7fcd740cb470 2023-07-08T12:39:34.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.990+0000 7fcd792af700 1 -- 172.21.15.12:0/3627926919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd740cb470 msgr2=0x7fcd7406ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:34.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.990+0000 7fcd792af700 1 --2- 172.21.15.12:0/3627926919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd740cb470 0x7fcd7406ba50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcd680098f0 tx=0x7fcd68009c00 comp rx=0 tx=0).stop 2023-07-08T12:39:34.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.990+0000 7fcd792af700 1 -- 172.21.15.12:0/3627926919 shutdown_connections 2023-07-08T12:39:34.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.990+0000 7fcd792af700 1 --2- 172.21.15.12:0/3627926919 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd7415c1e0 0x7fcd740caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:34.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.990+0000 7fcd792af700 1 --2- 172.21.15.12:0/3627926919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd740cb470 0x7fcd7406ba50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:34.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.990+0000 7fcd792af700 1 -- 172.21.15.12:0/3627926919 >> 172.21.15.12:0/3627926919 conn(0x7fcd7406e550 msgr2=0x7fcd7406e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:34.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.990+0000 7fcd792af700 1 -- 172.21.15.12:0/3627926919 shutdown_connections 2023-07-08T12:39:34.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.990+0000 7fcd792af700 1 -- 172.21.15.12:0/3627926919 wait complete. 2023-07-08T12:39:34.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.991+0000 7fcd792af700 1 Processor -- start 2023-07-08T12:39:34.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.991+0000 7fcd792af700 1 -- start start 2023-07-08T12:39:34.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd792af700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd7406bf90 0x7fcd740c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:34.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd792af700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd740cb470 0x7fcd74151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:34.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd792af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd7415c1e0 0x7fcd740c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:34.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd792af700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740c8fb0 con 0x7fcd7415c1e0 2023-07-08T12:39:34.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd792af700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740c9230 con 0x7fcd7406bf90 2023-07-08T12:39:34.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd792af700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740c9510 con 0x7fcd740cb470 2023-07-08T12:39:34.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd737fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd7415c1e0 0x7fcd740c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd72ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd7406bf90 0x7fcd740c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd737fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd7415c1e0 0x7fcd740c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51790/0 (socket says 172.21.15.12:51790) 2023-07-08T12:39:35.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd737fe700 1 -- 172.21.15.12:0/422945588 learned_addr learned my addr 172.21.15.12:0/422945588 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:35.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd727fc700 1 --2- 172.21.15.12:0/422945588 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd740cb470 0x7fcd74151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd737fe700 1 -- 172.21.15.12:0/422945588 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd740cb470 msgr2=0x7fcd74151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd737fe700 1 --2- 172.21.15.12:0/422945588 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd740cb470 0x7fcd74151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd737fe700 1 -- 172.21.15.12:0/422945588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd7406bf90 msgr2=0x7fcd740c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd737fe700 1 --2- 172.21.15.12:0/422945588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd7406bf90 0x7fcd740c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.992+0000 7fcd737fe700 1 -- 172.21.15.12:0/422945588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd68009550 con 0x7fcd7415c1e0 2023-07-08T12:39:35.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.993+0000 7fcd737fe700 1 --2- 172.21.15.12:0/422945588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd7415c1e0 0x7fcd740c8a70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00cbc0 tx=0x7fcd6c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:35.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.993+0000 7fcd5b7fe700 1 -- 172.21.15.12:0/422945588 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c003c00 con 0x7fcd7415c1e0 2023-07-08T12:39:35.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.993+0000 7fcd5b7fe700 1 -- 172.21.15.12:0/422945588 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c003d60 con 0x7fcd7415c1e0 2023-07-08T12:39:35.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.993+0000 7fcd5b7fe700 1 -- 172.21.15.12:0/422945588 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00aae0 con 0x7fcd7415c1e0 2023-07-08T12:39:35.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.993+0000 7fcd792af700 1 -- 172.21.15.12:0/422945588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd740c97f0 con 0x7fcd7415c1e0 2023-07-08T12:39:35.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.993+0000 7fcd792af700 1 -- 172.21.15.12:0/422945588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd740c9db0 con 0x7fcd7415c1e0 2023-07-08T12:39:35.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.994+0000 7fcd792af700 1 -- 172.21.15.12:0/422945588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd740c9fa0 con 0x7fcd7415c1e0 2023-07-08T12:39:35.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.994+0000 7fcd5b7fe700 1 -- 172.21.15.12:0/422945588 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd6c00ac40 con 0x7fcd7415c1e0 2023-07-08T12:39:35.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.995+0000 7fcd5b7fe700 1 --2- 172.21.15.12:0/422945588 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcd6006ace0 0x7fcd6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.995+0000 7fcd5b7fe700 1 -- 172.21.15.12:0/422945588 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcd6c0894e0 con 0x7fcd7415c1e0 2023-07-08T12:39:35.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.995+0000 7fcd72ffd700 1 --2- 172.21.15.12:0/422945588 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcd6006ace0 0x7fcd6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.996+0000 7fcd72ffd700 1 --2- 172.21.15.12:0/422945588 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcd6006ace0 0x7fcd6006d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcd64006b20 tx=0x7fcd64008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:35.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:34.999+0000 7fcd5b7fe700 1 -- 172.21.15.12:0/422945588 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcd6c0544e0 con 0x7fcd7415c1e0 2023-07-08T12:39:35.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.242+0000 7fcd792af700 1 -- 172.21.15.12:0/422945588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd7404ea60 con 0x7fcd7415c1e0 2023-07-08T12:39:35.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.242+0000 7fcd5b7fe700 1 -- 172.21.15.12:0/422945588 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4671 (secure 0 0 0) 0x7fcd6c057b00 con 0x7fcd7415c1e0 2023-07-08T12:39:35.245 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:35.245 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4937,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/1217176712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":1217176712},{"type":"v1","addr":"172.21.15.160:6836","nonce":1217176712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5215,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/396620950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":396620950},{"type":"v1","addr":"172.21.15.160:6837","nonce":396620950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6834/1725416152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1725416152},{"type":"v1","addr":"172.21.15.12:6834","nonce":1725416152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-08T12:39:31.213203+0000","modified":"2023-07-08T12:39:34.835529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5346},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5346":{"gid":5346,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.12:6835/590221748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":590221748},{"type":"v1","addr":"172.21.15.12:6835","nonce":590221748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-08T12:39:35.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.244+0000 7fcd792af700 1 -- 172.21.15.12:0/422945588 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcd6006ace0 msgr2=0x7fcd6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.244+0000 7fcd792af700 1 --2- 172.21.15.12:0/422945588 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcd6006ace0 0x7fcd6006d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcd64006b20 tx=0x7fcd64008040 comp rx=0 tx=0).stop 2023-07-08T12:39:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.244+0000 7fcd792af700 1 -- 172.21.15.12:0/422945588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd7415c1e0 msgr2=0x7fcd740c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.244+0000 7fcd792af700 1 --2- 172.21.15.12:0/422945588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd7415c1e0 0x7fcd740c8a70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00cbc0 tx=0x7fcd6c00ced0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.244+0000 7fcd792af700 1 -- 172.21.15.12:0/422945588 shutdown_connections 2023-07-08T12:39:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.244+0000 7fcd792af700 1 --2- 172.21.15.12:0/422945588 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fcd6006ace0 0x7fcd6006d190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.244+0000 7fcd792af700 1 --2- 172.21.15.12:0/422945588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd7415c1e0 0x7fcd740c8a70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.244+0000 7fcd792af700 1 --2- 172.21.15.12:0/422945588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd7406bf90 0x7fcd740c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.244+0000 7fcd792af700 1 --2- 172.21.15.12:0/422945588 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd740cb470 0x7fcd74151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.244+0000 7fcd792af700 1 -- 172.21.15.12:0/422945588 >> 172.21.15.12:0/422945588 conn(0x7fcd7406e550 msgr2=0x7fcd740b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:35.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.245+0000 7fcd792af700 1 -- 172.21.15.12:0/422945588 shutdown_connections 2023-07-08T12:39:35.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.245+0000 7fcd792af700 1 -- 172.21.15.12:0/422945588 wait complete. 2023-07-08T12:39:35.250 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-07-08T12:39:35.259 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-08T12:39:35.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.381+0000 7f4961d0c700 1 Processor -- start 2023-07-08T12:39:35.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.381+0000 7f4961d0c700 1 -- start start 2023-07-08T12:39:35.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.382+0000 7f4961d0c700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f495c0d2640 0x7f495c0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.382+0000 7f4961d0c700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495c0d1790 con 0x7f495c15c460 2023-07-08T12:39:35.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.382+0000 7f4961d0c700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495c0cd950 con 0x7f495c0cf8f0 2023-07-08T12:39:35.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.382+0000 7f4961d0c700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495c0cedd0 con 0x7f495c0d2640 2023-07-08T12:39:35.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.382+0000 7f495b7fe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f495c15c460 0x7f495c0cf1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39190/0 (socket says 172.21.15.12:39190) 2023-07-08T12:39:35.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.382+0000 7f495b7fe700 1 -- 172.21.15.12:0/3283192325 learned_addr learned my addr 172.21.15.12:0/3283192325 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:35.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.382+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962412888 0 0) 0x7f495c0d1790 con 0x7f495c15c460 2023-07-08T12:39:35.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.382+0000 7f495affd700 1 --2- 172.21.15.12:0/3283192325 >> v2:172.21.15.160:3300/0 conn(0x7f495c0d2640 0x7f495c0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49440035b0 con 0x7f495c15c460 2023-07-08T12:39:35.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 989679837 0 0) 0x7f49440035b0 con 0x7f495c15c460 2023-07-08T12:39:35.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 >> v2:172.21.15.160:3300/0 conn(0x7f495c0d2640 msgr2=0x7f495c0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:35.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 --2- 172.21.15.12:0/3283192325 >> v2:172.21.15.160:3300/0 conn(0x7f495c0d2640 0x7f495c0b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 >> v1:172.21.15.160:6789/0 conn(0x7f495c0cf8f0 legacy=0x7f495c15fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c0d14b0 con 0x7f495c15c460 2023-07-08T12:39:35.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f494c001670 con 0x7f495c15c460 2023-07-08T12:39:35.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495affd700 1 --2- 172.21.15.12:0/3283192325 >> v2:172.21.15.160:3300/0 conn(0x7f495c0d2640 0x7f495c0b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:35.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 >> v1:172.21.15.12:6789/0 conn(0x7f495c15c460 legacy=0x7f495c0cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 --2- 172.21.15.12:0/3283192325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4944003a00 0x7f4944003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 --2- 172.21.15.12:0/3283192325 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4944004850 0x7f4944004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 --2- 172.21.15.12:0/3283192325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4944005400 0x7f494400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c0cedd0 con 0x7f4944005400 2023-07-08T12:39:35.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494400c280 con 0x7f4944003a00 2023-07-08T12:39:35.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494400c440 con 0x7f4944004850 2023-07-08T12:39:35.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495bfff700 1 --2- 172.21.15.12:0/3283192325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4944005400 0x7f494400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495b7fe700 1 --2- 172.21.15.12:0/3283192325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4944003a00 0x7f4944003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495affd700 1 --2- 172.21.15.12:0/3283192325 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4944004850 0x7f4944004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495bfff700 1 -- 172.21.15.12:0/3283192325 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4944004850 msgr2=0x7f4944004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:35.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495bfff700 1 --2- 172.21.15.12:0/3283192325 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4944004850 0x7f4944004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495bfff700 1 -- 172.21.15.12:0/3283192325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4944003a00 msgr2=0x7f4944003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:35.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495bfff700 1 --2- 172.21.15.12:0/3283192325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4944003a00 0x7f4944003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495bfff700 1 -- 172.21.15.12:0/3283192325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440035b0 con 0x7f4944005400 2023-07-08T12:39:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.383+0000 7f495bfff700 1 -- 172.21.15.12:0/3283192325 reap_dead start 2023-07-08T12:39:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.384+0000 7f495bfff700 1 --2- 172.21.15.12:0/3283192325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4944005400 0x7f494400ba10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f495400c190 tx=0x7f4944005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a49e4d3499ceeb server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.384+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4954001db0 con 0x7f4944005400 2023-07-08T12:39:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.384+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495400ccf0 con 0x7f4944005400 2023-07-08T12:39:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.384+0000 7f495a7fc700 1 -- 172.21.15.12:0/3283192325 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4954003c40 con 0x7f4944005400 2023-07-08T12:39:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.384+0000 7f4961d0c700 1 -- 172.21.15.12:0/3283192325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4944005400 msgr2=0x7f494400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.384+0000 7f4961d0c700 1 --2- 172.21.15.12:0/3283192325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4944005400 0x7f494400ba10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f495400c190 tx=0x7f4944005880 comp rx=0 tx=0).stop 2023-07-08T12:39:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.384+0000 7f4961d0c700 1 -- 172.21.15.12:0/3283192325 shutdown_connections 2023-07-08T12:39:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.384+0000 7f4961d0c700 1 --2- 172.21.15.12:0/3283192325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4944005400 0x7f494400ba10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.384+0000 7f4961d0c700 1 -- 172.21.15.12:0/3283192325 >> 172.21.15.12:0/3283192325 conn(0x7f495c071660 msgr2=0x7f495c071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.384+0000 7f4961d0c700 1 -- 172.21.15.12:0/3283192325 shutdown_connections 2023-07-08T12:39:35.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.384+0000 7f4961d0c700 1 -- 172.21.15.12:0/3283192325 wait complete. 2023-07-08T12:39:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.386+0000 7f4961d0c700 1 Processor -- start 2023-07-08T12:39:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.386+0000 7f4961d0c700 1 -- start start 2023-07-08T12:39:35.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f4961d0c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4944005400 0x7f495c14a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f4961d0c700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f495c15ef00 0x7f495c0cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f4961d0c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f495c15f370 0x7f495c0cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f4961d0c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c15eae0 con 0x7f495c15f370 2023-07-08T12:39:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f4961d0c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c15de60 con 0x7f4944005400 2023-07-08T12:39:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f4961d0c700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495c15e140 con 0x7f495c15ef00 2023-07-08T12:39:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f495bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f495c15f370 0x7f495c0cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f495bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f495c15f370 0x7f495c0cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51810/0 (socket says 172.21.15.12:51810) 2023-07-08T12:39:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f495bfff700 1 -- 172.21.15.12:0/2477586652 learned_addr learned my addr 172.21.15.12:0/2477586652 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f495b7fe700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4944005400 0x7f495c14a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f495affd700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f495c15ef00 0x7f495c0cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f495bfff700 1 -- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f495c15ef00 msgr2=0x7f495c0cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f495bfff700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f495c15ef00 0x7f495c0cbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f495bfff700 1 -- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4944005400 msgr2=0x7f495c14a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f495bfff700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4944005400 0x7f495c14a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.387+0000 7f495bfff700 1 -- 172.21.15.12:0/2477586652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c15e420 con 0x7f495c15f370 2023-07-08T12:39:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.388+0000 7f495bfff700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f495c15f370 0x7f495c0cc360 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4954005040 tx=0x7f49540014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.388+0000 7f4943fff700 1 -- 172.21.15.12:0/2477586652 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49540052d0 con 0x7f495c15f370 2023-07-08T12:39:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.388+0000 7f4943fff700 1 -- 172.21.15.12:0/2477586652 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4954005430 con 0x7f495c15f370 2023-07-08T12:39:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.388+0000 7f4943fff700 1 -- 172.21.15.12:0/2477586652 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49540057a0 con 0x7f495c15f370 2023-07-08T12:39:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.388+0000 7f4961d0c700 1 -- 172.21.15.12:0/2477586652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495c15e700 con 0x7f495c15f370 2023-07-08T12:39:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.388+0000 7f4961d0c700 1 -- 172.21.15.12:0/2477586652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495c14a990 con 0x7f495c15f370 2023-07-08T12:39:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.389+0000 7f4961d0c700 1 -- 172.21.15.12:0/2477586652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495c15ec20 con 0x7f495c15f370 2023-07-08T12:39:35.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.389+0000 7f4943fff700 1 -- 172.21.15.12:0/2477586652 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f495401d480 con 0x7f495c15f370 2023-07-08T12:39:35.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.390+0000 7f4943fff700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f494806ace0 0x7f494806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.390+0000 7f4943fff700 1 -- 172.21.15.12:0/2477586652 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f495408a2b0 con 0x7f495c15f370 2023-07-08T12:39:35.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.390+0000 7f495b7fe700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f494806ace0 0x7f494806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.391+0000 7f495b7fe700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f494806ace0 0x7f494806d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f494c000c00 tx=0x7f494c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:35.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.394+0000 7f4943fff700 1 -- 172.21.15.12:0/2477586652 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f495c15ec20 con 0x7f495c15f370 2023-07-08T12:39:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.595+0000 7f4961d0c700 1 -- 172.21.15.12:0/2477586652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f495c15ec20 con 0x7f495c15f370 2023-07-08T12:39:35.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.596+0000 7f4943fff700 1 -- 172.21.15.12:0/2477586652 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18488 (secure 0 0 0) 0x7f495c15ec20 con 0x7f495c15f370 2023-07-08T12:39:35.599 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:35.599 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":42,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","created":"2023-07-08T12:37:42.501417+0000","modified":"2023-07-08T12:39:32.825271+0000","last_up_change":"2023-07-08T12:37:54.322339+0000","last_in_change":"2023-07-08T12:37:45.094491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-08T12:37:56.195631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-08T12:38:01.187530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-07-08T12:39:29.579249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-07-08T12:39:30.175950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f72a321b-c04e-4997-a4a6-3b1feb214dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6817","nonce":2950255941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6819","nonce":2950255941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6823","nonce":2950255941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":2950255941},{"type":"v1","addr":"172.21.15.12:6821","nonce":2950255941}]},"public_addr":"172.21.15.12:6817/2950255941","cluster_addr":"172.21.15.12:6819/2950255941","heartbeat_back_addr":"172.21.15.12:6823/2950255941","heartbeat_front_addr":"172.21.15.12:6821/2950255941","state":["exists","up"]},{"osd":1,"uuid":"9dacfd32-474b-426e-afd7-0cbeb01f0381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6801","nonce":911521993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6803","nonce":911521993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6807","nonce":911521993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":911521993},{"type":"v1","addr":"172.21.15.12:6805","nonce":911521993}]},"public_addr":"172.21.15.12:6801/911521993","cluster_addr":"172.21.15.12:6803/911521993","heartbeat_back_addr":"172.21.15.12:6807/911521993","heartbeat_front_addr":"172.21.15.12:6805/911521993","state":["exists","up"]},{"osd":2,"uuid":"10b4375f-fca4-4632-b24b-7766da4949ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6809","nonce":720039673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6811","nonce":720039673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6815","nonce":720039673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":720039673},{"type":"v1","addr":"172.21.15.12:6813","nonce":720039673}]},"public_addr":"172.21.15.12:6809/720039673","cluster_addr":"172.21.15.12:6811/720039673","heartbeat_back_addr":"172.21.15.12:6815/720039673","heartbeat_front_addr":"172.21.15.12:6813/720039673","state":["exists","up"]},{"osd":3,"uuid":"98684225-d5af-4e14-9deb-32a335b2f9e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6825","nonce":1815509748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6827","nonce":1815509748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6831","nonce":1815509748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1815509748},{"type":"v1","addr":"172.21.15.12:6829","nonce":1815509748}]},"public_addr":"172.21.15.12:6825/1815509748","cluster_addr":"172.21.15.12:6827/1815509748","heartbeat_back_addr":"172.21.15.12:6831/1815509748","heartbeat_front_addr":"172.21.15.12:6829/1815509748","state":["exists","up"]},{"osd":4,"uuid":"d059f27b-b32d-4f7b-a2f1-18b71a396093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6810","nonce":1968203867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6813","nonce":1968203867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6820","nonce":1968203867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":1968203867},{"type":"v1","addr":"172.21.15.160:6817","nonce":1968203867}]},"public_addr":"172.21.15.160:6810/1968203867","cluster_addr":"172.21.15.160:6813/1968203867","heartbeat_back_addr":"172.21.15.160:6820/1968203867","heartbeat_front_addr":"172.21.15.160:6817/1968203867","state":["exists","up"]},{"osd":5,"uuid":"92528dc3-7627-4bad-9575-9ceda9430855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6823","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6825","nonce":2886140325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6827","nonce":2886140325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6831","nonce":2886140325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2886140325},{"type":"v1","addr":"172.21.15.160:6829","nonce":2886140325}]},"public_addr":"172.21.15.160:6825/2886140325","cluster_addr":"172.21.15.160:6827/2886140325","heartbeat_back_addr":"172.21.15.160:6831/2886140325","heartbeat_front_addr":"172.21.15.160:6829/2886140325","state":["exists","up"]},{"osd":6,"uuid":"b4ef7b59-901a-41bd-8705-afeab65c4b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6801","nonce":2675224151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6803","nonce":2675224151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6807","nonce":2675224151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2675224151},{"type":"v1","addr":"172.21.15.160:6805","nonce":2675224151}]},"public_addr":"172.21.15.160:6801/2675224151","cluster_addr":"172.21.15.160:6803/2675224151","heartbeat_back_addr":"172.21.15.160:6807/2675224151","heartbeat_front_addr":"172.21.15.160:6805/2675224151","state":["exists","up"]},{"osd":7,"uuid":"06bf1e54-02a7-48c8-a72e-9364b2c575d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6811","nonce":4149495008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6816","nonce":4149495008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6824","nonce":4149495008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6819","nonce":4149495008},{"type":"v1","addr":"172.21.15.160:6821","nonce":4149495008}]},"public_addr":"172.21.15.160:6811/4149495008","cluster_addr":"172.21.15.160:6816/4149495008","heartbeat_back_addr":"172.21.15.160:6824/4149495008","heartbeat_front_addr":"172.21.15.160:6821/4149495008","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.275306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.290028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.339759+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.273748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.358781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.353455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.270064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-08T12:37:50.389849+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-08T12:39:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.598+0000 7f4961d0c700 1 -- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f494806ace0 msgr2=0x7f494806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.598+0000 7f4961d0c700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f494806ace0 0x7f494806d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f494c000c00 tx=0x7f494c009040 comp rx=0 tx=0).stop 2023-07-08T12:39:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.598+0000 7f4961d0c700 1 -- 172.21.15.12:0/2477586652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f495c15f370 msgr2=0x7f495c0cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.598+0000 7f4961d0c700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f495c15f370 0x7f495c0cc360 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4954005040 tx=0x7f49540014a0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.598+0000 7f4961d0c700 1 -- 172.21.15.12:0/2477586652 shutdown_connections 2023-07-08T12:39:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.598+0000 7f4961d0c700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f494806ace0 0x7f494806d190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.598+0000 7f4961d0c700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f495c15f370 0x7f495c0cc360 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.598+0000 7f4961d0c700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4944005400 0x7f495c14a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.598+0000 7f4961d0c700 1 --2- 172.21.15.12:0/2477586652 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f495c15ef00 0x7f495c0cbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.598+0000 7f4961d0c700 1 -- 172.21.15.12:0/2477586652 >> 172.21.15.12:0/2477586652 conn(0x7f495c071660 msgr2=0x7f495c0c86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:35.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.599+0000 7f4961d0c700 1 -- 172.21.15.12:0/2477586652 shutdown_connections 2023-07-08T12:39:35.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.599+0000 7f4961d0c700 1 -- 172.21.15.12:0/2477586652 wait complete. 2023-07-08T12:39:35.613 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-08T12:39:35.613 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-08T12:39:35.676 INFO:tasks.ceph.mds.a.smithi012.stderr:2023-07-08T12:39:35.674+0000 7f47834bc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-08T12:39:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.735+0000 7fb148c3f700 1 Processor -- start 2023-07-08T12:39:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.735+0000 7fb148c3f700 1 -- start start 2023-07-08T12:39:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.736+0000 7fb148c3f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb144159c30 0x7fb1440cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.736+0000 7fb148c3f700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb144069f30 0x7fb14406a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.736+0000 7fb148c3f700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14415c890 con 0x7fb144160790 2023-07-08T12:39:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.736+0000 7fb148c3f700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb14415c730 con 0x7fb144069f30 2023-07-08T12:39:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.736+0000 7fb148c3f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440cf590 con 0x7fb144159c30 2023-07-08T12:39:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.737+0000 7fb142d9d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb144069f30 0x7fb14406a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.737+0000 7fb141d9b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb144160790 0x7fb14406ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39212/0 (socket says 172.21.15.12:39212) 2023-07-08T12:39:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.737+0000 7fb141d9b700 1 -- 172.21.15.12:0/1943673415 learned_addr learned my addr 172.21.15.12:0/1943673415 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.737+0000 7fb142d9d700 1 -- 172.21.15.12:0/1943673415 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb144159c30 msgr2=0x7fb1440cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.737+0000 7fb142d9d700 1 --2- 172.21.15.12:0/1943673415 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb144159c30 0x7fb1440cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.737+0000 7fb142d9d700 1 -- 172.21.15.12:0/1943673415 >> v1:172.21.15.12:6789/0 conn(0x7fb144160790 legacy=0x7fb14406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.737+0000 7fb142d9d700 1 -- 172.21.15.12:0/1943673415 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14406f620 con 0x7fb144069f30 2023-07-08T12:39:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.737+0000 7fb14159a700 1 -- 172.21.15.12:0/1943673415 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1012529041 0 0) 0x7fb14415c890 con 0x7fb144160790 2023-07-08T12:39:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.737+0000 7fb142d9d700 1 --2- 172.21.15.12:0/1943673415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb144069f30 0x7fb14406a3a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009dc0 tx=0x7fb13c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559d31f1300062ba server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.737+0000 7fb14159a700 1 -- 172.21.15.12:0/1943673415 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00e070 con 0x7fb144069f30 2023-07-08T12:39:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.737+0000 7fb14159a700 1 -- 172.21.15.12:0/1943673415 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13c00bb90 con 0x7fb144069f30 2023-07-08T12:39:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.737+0000 7fb14159a700 1 -- 172.21.15.12:0/1943673415 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c012430 con 0x7fb144069f30 2023-07-08T12:39:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.738+0000 7fb148c3f700 1 -- 172.21.15.12:0/1943673415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb144069f30 msgr2=0x7fb14406a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.738+0000 7fb148c3f700 1 --2- 172.21.15.12:0/1943673415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb144069f30 0x7fb14406a3a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009dc0 tx=0x7fb13c00b330 comp rx=0 tx=0).stop 2023-07-08T12:39:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.738+0000 7fb148c3f700 1 -- 172.21.15.12:0/1943673415 shutdown_connections 2023-07-08T12:39:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.738+0000 7fb148c3f700 1 --2- 172.21.15.12:0/1943673415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb144069f30 0x7fb14406a3a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.738+0000 7fb148c3f700 1 --2- 172.21.15.12:0/1943673415 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb144159c30 0x7fb1440cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.738+0000 7fb148c3f700 1 -- 172.21.15.12:0/1943673415 >> 172.21.15.12:0/1943673415 conn(0x7fb1440cc290 msgr2=0x7fb1440cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.738+0000 7fb148c3f700 1 -- 172.21.15.12:0/1943673415 shutdown_connections 2023-07-08T12:39:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.738+0000 7fb148c3f700 1 -- 172.21.15.12:0/1943673415 wait complete. 2023-07-08T12:39:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.739+0000 7fb148c3f700 1 Processor -- start 2023-07-08T12:39:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.739+0000 7fb148c3f700 1 -- start start 2023-07-08T12:39:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb148c3f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb144069f30 0x7fb144142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb148c3f700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb144159c30 0x7fb1440ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb148c3f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb144160790 0x7fb1440c2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb148c3f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440abdf0 con 0x7fb144069f30 2023-07-08T12:39:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb148c3f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440ad680 con 0x7fb144160790 2023-07-08T12:39:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb148c3f700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440b54d0 con 0x7fb144159c30 2023-07-08T12:39:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb14259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb144069f30 0x7fb144142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb14259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb144069f30 0x7fb144142fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51820/0 (socket says 172.21.15.12:51820) 2023-07-08T12:39:35.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb14259c700 1 -- 172.21.15.12:0/3685467548 learned_addr learned my addr 172.21.15.12:0/3685467548 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:35.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb141d9b700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb144159c30 0x7fb1440ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb142d9d700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb144160790 0x7fb1440c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb14259c700 1 -- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb144159c30 msgr2=0x7fb1440ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb14259c700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb144159c30 0x7fb1440ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb14259c700 1 -- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb144160790 msgr2=0x7fb1440c2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb14259c700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb144160790 0x7fb1440c2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.740+0000 7fb14259c700 1 -- 172.21.15.12:0/3685467548 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c009a70 con 0x7fb144069f30 2023-07-08T12:39:35.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.741+0000 7fb14259c700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb144069f30 0x7fb144142fb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb13400b530 tx=0x7fb13400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:35.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.741+0000 7fb13affd700 1 -- 172.21.15.12:0/3685467548 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134011830 con 0x7fb144069f30 2023-07-08T12:39:35.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.741+0000 7fb13affd700 1 -- 172.21.15.12:0/3685467548 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb134011990 con 0x7fb144069f30 2023-07-08T12:39:35.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.741+0000 7fb13affd700 1 -- 172.21.15.12:0/3685467548 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134011d40 con 0x7fb144069f30 2023-07-08T12:39:35.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.741+0000 7fb148c3f700 1 -- 172.21.15.12:0/3685467548 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb144145b80 con 0x7fb144069f30 2023-07-08T12:39:35.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.741+0000 7fb148c3f700 1 -- 172.21.15.12:0/3685467548 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb144156870 con 0x7fb144069f30 2023-07-08T12:39:35.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.742+0000 7fb148c3f700 1 -- 172.21.15.12:0/3685467548 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb14404ea60 con 0x7fb144069f30 2023-07-08T12:39:35.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.742+0000 7fb13affd700 1 -- 172.21.15.12:0/3685467548 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb13401a400 con 0x7fb144069f30 2023-07-08T12:39:35.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.743+0000 7fb13affd700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb12406ace0 0x7fb12406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:35.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.743+0000 7fb13affd700 1 -- 172.21.15.12:0/3685467548 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb134089590 con 0x7fb144069f30 2023-07-08T12:39:35.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.743+0000 7fb141d9b700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb12406ace0 0x7fb12406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:35.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.743+0000 7fb141d9b700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb12406ace0 0x7fb12406d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb12c007fd0 tx=0x7fb12c007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:35.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.747+0000 7fb13affd700 1 -- 172.21.15.12:0/3685467548 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb1340545b0 con 0x7fb144069f30 2023-07-08T12:39:35.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.941+0000 7fb148c3f700 1 -- 172.21.15.12:0/3685467548 --> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb144160c90 con 0x7fb12406ace0 2023-07-08T12:39:35.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.950+0000 7fb13affd700 1 -- 172.21.15.12:0/3685467548 <== mgr.4103 v2:172.21.15.160:6832/550875184 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+298563 (secure 0 0 0) 0x7fb12c001370 con 0x7fb12406ace0 2023-07-08T12:39:35.955 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:35.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.953+0000 7fb148c3f700 1 -- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb12406ace0 msgr2=0x7fb12406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.953+0000 7fb148c3f700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb12406ace0 0x7fb12406d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb12c007fd0 tx=0x7fb12c007de0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.953+0000 7fb148c3f700 1 -- 172.21.15.12:0/3685467548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb144069f30 msgr2=0x7fb144142fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:35.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.953+0000 7fb148c3f700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb144069f30 0x7fb144142fb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb13400b530 tx=0x7fb13400b840 comp rx=0 tx=0).stop 2023-07-08T12:39:35.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.953+0000 7fb148c3f700 1 -- 172.21.15.12:0/3685467548 shutdown_connections 2023-07-08T12:39:35.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.953+0000 7fb148c3f700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb12406ace0 0x7fb12406d190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.953+0000 7fb148c3f700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb144160790 0x7fb1440c2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.953+0000 7fb148c3f700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb144069f30 0x7fb144142fb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.953+0000 7fb148c3f700 1 --2- 172.21.15.12:0/3685467548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb144159c30 0x7fb1440ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:35.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.953+0000 7fb148c3f700 1 -- 172.21.15.12:0/3685467548 >> 172.21.15.12:0/3685467548 conn(0x7fb1440cc290 msgr2=0x7fb14406ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:35.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.953+0000 7fb148c3f700 1 -- 172.21.15.12:0/3685467548 shutdown_connections 2023-07-08T12:39:35.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:35.953+0000 7fb148c3f700 1 -- 172.21.15.12:0/3685467548 wait complete. 2023-07-08T12:39:35.959 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-07-08T12:39:35.967 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2023-07-08T12:39:34.162574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":64,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220224,"kb_used_data":4608,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754496,"statfs":{"total":773094113280,"available":772868603904,"internally_reserved":0,"allocated":4718592,"data_stored":2710516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.376723"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:29.780305+0000","last_change":"2023-07-08T12:39:29.780305+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.780305+0000","last_undegraded":"2023-07-08T12:39:29.780305+0000","last_fullsized":"2023-07-08T12:39:29.780305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.787913+0000","last_change":"2023-07-08T12:39:29.787913+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.787913+0000","last_undegraded":"2023-07-08T12:39:29.787913+0000","last_fullsized":"2023-07-08T12:39:29.787913+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.802733+0000","last_change":"2023-07-08T12:39:29.802733+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.802733+0000","last_undegraded":"2023-07-08T12:39:29.802733+0000","last_fullsized":"2023-07-08T12:39:29.802733+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.795111+0000","last_change":"2023-07-08T12:39:29.795111+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.795111+0000","last_undegraded":"2023-07-08T12:39:29.795111+0000","last_fullsized":"2023-07-08T12:39:29.795111+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:29.780305+0000","last_change":"2023-07-08T12:39:29.780305+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.780305+0000","last_undegraded":"2023-07-08T12:39:29.780305+0000","last_fullsized":"2023-07-08T12:39:29.780305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.788629+0000","last_change":"2023-07-08T12:39:29.788629+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.788629+0000","last_undegraded":"2023-07-08T12:39:29.788629+0000","last_fullsized":"2023-07-08T12:39:29.788629+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.796039+0000","last_change":"2023-07-08T12:39:29.796039+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.796039+0000","last_undegraded":"2023-07-08T12:39:29.796039+0000","last_fullsized":"2023-07-08T12:39:29.796039+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.791084+0000","last_change":"2023-07-08T12:39:29.791084+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.791084+0000","last_undegraded":"2023-07-08T12:39:29.791084+0000","last_fullsized":"2023-07-08T12:39:29.791084+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.787989+0000","last_change":"2023-07-08T12:39:29.787989+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.787989+0000","last_undegraded":"2023-07-08T12:39:29.787989+0000","last_fullsized":"2023-07-08T12:39:29.787989+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.790693+0000","last_change":"2023-07-08T12:39:29.790693+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.790693+0000","last_undegraded":"2023-07-08T12:39:29.790693+0000","last_fullsized":"2023-07-08T12:39:29.790693+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.797741+0000","last_change":"2023-07-08T12:39:29.797741+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.797741+0000","last_undegraded":"2023-07-08T12:39:29.797741+0000","last_fullsized":"2023-07-08T12:39:29.797741+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:29.780305+0000","last_change":"2023-07-08T12:39:29.780305+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.780305+0000","last_undegraded":"2023-07-08T12:39:29.780305+0000","last_fullsized":"2023-07-08T12:39:29.780305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:29.780305+0000","last_change":"2023-07-08T12:39:29.780305+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.780305+0000","last_undegraded":"2023-07-08T12:39:29.780305+0000","last_fullsized":"2023-07-08T12:39:29.780305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.795543+0000","last_change":"2023-07-08T12:39:29.795543+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.795543+0000","last_undegraded":"2023-07-08T12:39:29.795543+0000","last_fullsized":"2023-07-08T12:39:29.795543+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:29.780305+0000","last_change":"2023-07-08T12:39:29.780305+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.780305+0000","last_undegraded":"2023-07-08T12:39:29.780305+0000","last_fullsized":"2023-07-08T12:39:29.780305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.790609+0000","last_change":"2023-07-08T12:39:29.790609+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.790609+0000","last_undegraded":"2023-07-08T12:39:29.790609+0000","last_fullsized":"2023-07-08T12:39:29.790609+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:29.780305+0000","last_change":"2023-07-08T12:39:29.780305+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.780305+0000","last_undegraded":"2023-07-08T12:39:29.780305+0000","last_fullsized":"2023-07-08T12:39:29.780305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.788911+0000","last_change":"2023-07-08T12:39:29.788911+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.788911+0000","last_undegraded":"2023-07-08T12:39:29.788911+0000","last_fullsized":"2023-07-08T12:39:29.788911+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.796434+0000","last_change":"2023-07-08T12:39:29.796434+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.796434+0000","last_undegraded":"2023-07-08T12:39:29.796434+0000","last_fullsized":"2023-07-08T12:39:29.796434+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:29.780305+0000","last_change":"2023-07-08T12:39:29.780305+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.780305+0000","last_undegraded":"2023-07-08T12:39:29.780305+0000","last_fullsized":"2023-07-08T12:39:29.780305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:29.780305+0000","last_change":"2023-07-08T12:39:29.780305+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.780305+0000","last_undegraded":"2023-07-08T12:39:29.780305+0000","last_fullsized":"2023-07-08T12:39:29.780305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.796028+0000","last_change":"2023-07-08T12:39:29.796028+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.796028+0000","last_undegraded":"2023-07-08T12:39:29.796028+0000","last_fullsized":"2023-07-08T12:39:29.796028+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.790265+0000","last_change":"2023-07-08T12:39:29.790265+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.790265+0000","last_undegraded":"2023-07-08T12:39:29.790265+0000","last_fullsized":"2023-07-08T12:39:29.790265+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.795181+0000","last_change":"2023-07-08T12:39:29.795181+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.795181+0000","last_undegraded":"2023-07-08T12:39:29.795181+0000","last_fullsized":"2023-07-08T12:39:29.795181+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.790218+0000","last_change":"2023-07-08T12:39:29.790218+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.790218+0000","last_undegraded":"2023-07-08T12:39:29.790218+0000","last_fullsized":"2023-07-08T12:39:29.790218+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-07-08T12:39:29.803267+0000","last_change":"2023-07-08T12:38:03.160177+0000","last_active":"2023-07-08T12:39:29.803267+0000","last_peered":"2023-07-08T12:39:29.803267+0000","last_clean":"2023-07-08T12:39:29.803267+0000","last_became_active":"2023-07-08T12:38:03.159864+0000","last_became_peered":"2023-07-08T12:38:03.159864+0000","last_unstale":"2023-07-08T12:39:29.803267+0000","last_undegraded":"2023-07-08T12:39:29.803267+0000","last_fullsized":"2023-07-08T12:39:29.803267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:50:55.275344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:29.780305+0000","last_change":"2023-07-08T12:39:29.780305+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.780305+0000","last_undegraded":"2023-07-08T12:39:29.780305+0000","last_fullsized":"2023-07-08T12:39:29.780305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-07-08T12:39:29.796819+0000","last_change":"2023-07-08T12:38:02.384639+0000","last_active":"2023-07-08T12:39:29.796819+0000","last_peered":"2023-07-08T12:39:29.796819+0000","last_clean":"2023-07-08T12:39:29.796819+0000","last_became_active":"2023-07-08T12:38:02.384372+0000","last_became_peered":"2023-07-08T12:38:02.384372+0000","last_unstale":"2023-07-08T12:39:29.796819+0000","last_undegraded":"2023-07-08T12:39:29.796819+0000","last_fullsized":"2023-07-08T12:39:29.796819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T16:59:06.651005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.797295+0000","last_change":"2023-07-08T12:39:29.797295+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.797295+0000","last_undegraded":"2023-07-08T12:39:29.797295+0000","last_fullsized":"2023-07-08T12:39:29.797295+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-07-08T12:39:24.676151+0000","last_change":"2023-07-08T12:38:03.212466+0000","last_active":"2023-07-08T12:39:24.676151+0000","last_peered":"2023-07-08T12:39:24.676151+0000","last_clean":"2023-07-08T12:39:24.676151+0000","last_became_active":"2023-07-08T12:38:03.212182+0000","last_became_peered":"2023-07-08T12:38:03.212182+0000","last_unstale":"2023-07-08T12:39:24.676151+0000","last_undegraded":"2023-07-08T12:39:24.676151+0000","last_fullsized":"2023-07-08T12:39:24.676151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:16:31.355946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.796856+0000","last_change":"2023-07-08T12:39:29.796856+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.796856+0000","last_undegraded":"2023-07-08T12:39:29.796856+0000","last_fullsized":"2023-07-08T12:39:29.796856+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-07-08T12:39:29.796722+0000","last_change":"2023-07-08T12:38:02.380175+0000","last_active":"2023-07-08T12:39:29.796722+0000","last_peered":"2023-07-08T12:39:29.796722+0000","last_clean":"2023-07-08T12:39:29.796722+0000","last_became_active":"2023-07-08T12:38:02.379867+0000","last_became_peered":"2023-07-08T12:38:02.379867+0000","last_unstale":"2023-07-08T12:39:29.796722+0000","last_undegraded":"2023-07-08T12:39:29.796722+0000","last_fullsized":"2023-07-08T12:39:29.796722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:37:36.708048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-07-08T12:39:29.798262+0000","last_change":"2023-07-08T12:37:58.222371+0000","last_active":"2023-07-08T12:39:29.798262+0000","last_peered":"2023-07-08T12:39:29.798262+0000","last_clean":"2023-07-08T12:39:29.798262+0000","last_became_active":"2023-07-08T12:37:58.221381+0000","last_became_peered":"2023-07-08T12:37:58.221381+0000","last_unstale":"2023-07-08T12:39:29.798262+0000","last_undegraded":"2023-07-08T12:39:29.798262+0000","last_fullsized":"2023-07-08T12:39:29.798262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:37:56.327863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:37:56.327863+0000","last_clean_scrub_stamp":"2023-07-08T12:37:56.327863+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:09:15.864662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.802726+0000","last_change":"2023-07-08T12:39:29.802726+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.802726+0000","last_undegraded":"2023-07-08T12:39:29.802726+0000","last_fullsized":"2023-07-08T12:39:29.802726+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-07-08T12:39:29.798452+0000","last_change":"2023-07-08T12:38:02.385193+0000","last_active":"2023-07-08T12:39:29.798452+0000","last_peered":"2023-07-08T12:39:29.798452+0000","last_clean":"2023-07-08T12:39:29.798452+0000","last_became_active":"2023-07-08T12:38:02.384865+0000","last_became_peered":"2023-07-08T12:38:02.384865+0000","last_unstale":"2023-07-08T12:39:29.798452+0000","last_undegraded":"2023-07-08T12:39:29.798452+0000","last_fullsized":"2023-07-08T12:39:29.798452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:44:45.403478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.802707+0000","last_change":"2023-07-08T12:39:29.802707+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.802707+0000","last_undegraded":"2023-07-08T12:39:29.802707+0000","last_fullsized":"2023-07-08T12:39:29.802707+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-07-08T12:39:29.798360+0000","last_change":"2023-07-08T12:38:03.160344+0000","last_active":"2023-07-08T12:39:29.798360+0000","last_peered":"2023-07-08T12:39:29.798360+0000","last_clean":"2023-07-08T12:39:29.798360+0000","last_became_active":"2023-07-08T12:38:03.160009+0000","last_became_peered":"2023-07-08T12:38:03.160009+0000","last_unstale":"2023-07-08T12:39:29.798360+0000","last_undegraded":"2023-07-08T12:39:29.798360+0000","last_fullsized":"2023-07-08T12:39:29.798360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:37:01.522872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.787946+0000","last_change":"2023-07-08T12:39:29.787946+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.787946+0000","last_undegraded":"2023-07-08T12:39:29.787946+0000","last_fullsized":"2023-07-08T12:39:29.787946+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-07-08T12:39:29.803444+0000","last_change":"2023-07-08T12:38:02.385821+0000","last_active":"2023-07-08T12:39:29.803444+0000","last_peered":"2023-07-08T12:39:29.803444+0000","last_clean":"2023-07-08T12:39:29.803444+0000","last_became_active":"2023-07-08T12:38:02.385512+0000","last_became_peered":"2023-07-08T12:38:02.385512+0000","last_unstale":"2023-07-08T12:39:29.803444+0000","last_undegraded":"2023-07-08T12:39:29.803444+0000","last_fullsized":"2023-07-08T12:39:29.803444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:14:55.191170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-08T12:39:30.782334+0000","last_change":"2023-07-08T12:39:30.782334+0000","last_active":"2023-07-08T12:39:30.782334+0000","last_peered":"2023-07-08T12:39:30.782334+0000","last_clean":"2023-07-08T12:39:30.782334+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:30.782334+0000","last_undegraded":"2023-07-08T12:39:30.782334+0000","last_fullsized":"2023-07-08T12:39:30.782334+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2023-07-08T12:39:29.795088+0000","last_change":"2023-07-08T12:39:29.795088+0000","last_active":"2023-07-08T12:39:29.780305+0000","last_peered":"2023-07-08T12:39:29.780305+0000","last_clean":"2023-07-08T12:39:29.780305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-08T12:39:29.795088+0000","last_undegraded":"2023-07-08T12:39:29.795088+0000","last_fullsized":"2023-07-08T12:39:29.795088+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-07-08T12:39:29.787445+0000","last_change":"2023-07-08T12:38:02.385388+0000","last_active":"2023-07-08T12:39:29.787445+0000","last_peered":"2023-07-08T12:39:29.787445+0000","last_clean":"2023-07-08T12:39:29.787445+0000","last_became_active":"2023-07-08T12:38:02.385110+0000","last_became_peered":"2023-07-08T12:38:02.385110+0000","last_unstale":"2023-07-08T12:39:29.787445+0000","last_undegraded":"2023-07-08T12:39:29.787445+0000","last_fullsized":"2023-07-08T12:39:29.787445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:17:08.289067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":46,"acting":46,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":1044480,"data_stored":786458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.67800000000000005}]},{"osd":1,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54300000000000004}]},{"osd":2,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.65800000000000003}]},{"osd":3,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.71499999999999997}]},{"osd":4,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.73699999999999999}]},{"osd":5,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.70299999999999996}]},{"osd":6,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":189600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60399999999999998}]},{"osd":1,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.52100000000000002}]},{"osd":2,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":11.653,"5min":11.653,"15min":11.653},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48499999999999999}]},{"osd":3,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.503}]},{"osd":4,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56899999999999995}]},{"osd":5,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.63400000000000001}]},{"osd":7,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":440,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":450560,"data_stored":196090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.38500000000000001}]},{"osd":2,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.33800000000000002}]},{"osd":3,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.35799999999999998}]},{"osd":4,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55300000000000005}]},{"osd":5,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58399999999999996}]},{"osd":6,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.629},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":1032192,"data_stored":779968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":10.974,"5min":10.974,"15min":10.974},"last":0.221}]},{"osd":2,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.307},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.32300000000000001}]},{"osd":3,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.34599999999999997}]},{"osd":4,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.992000000000001,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.68999999999999995}]},{"osd":6,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":12.029,"5min":12.029,"15min":12.029},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.64100000000000001}]},{"osd":7,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":11.941000000000001,"5min":11.941000000000001,"15min":11.941000000000001},"last":0.62}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":189600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.28899999999999998}]},{"osd":1,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.39100000000000001}]},{"osd":3,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.375},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.41299999999999998}]},{"osd":4,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.76600000000000001}]},{"osd":6,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.74399999999999999}]},{"osd":7,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.63200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":189600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.48399999999999999}]},{"osd":1,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.45600000000000002}]},{"osd":2,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.433}]},{"osd":4,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.77400000000000002}]},{"osd":6,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.70199999999999996}]},{"osd":7,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.75700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":189600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.5}]},{"osd":1,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.40600000000000003}]},{"osd":3,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57599999999999996}]},{"osd":5,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.206,"5min":3.206,"15min":3.206},"last":0.378}]},{"osd":6,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56200000000000006}]},{"osd":7,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.859,"5min":3.859,"15min":3.859},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.60299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":189600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.40200000000000002}]},{"osd":1,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.49099999999999999}]},{"osd":2,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65100000000000002}]},{"osd":3,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.61699999999999999}]},{"osd":4,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.443}]},{"osd":6,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52700000000000002}]},{"osd":7,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68700000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-08T12:39:35.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-08T12:39:36.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.129+0000 7f42c7e24700 1 Processor -- start 2023-07-08T12:39:36.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.130+0000 7f42c7e24700 1 -- start start 2023-07-08T12:39:36.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.130+0000 7f42c7e24700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f42c015c740 0x7f42c00cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.130+0000 7f42c7e24700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f42c0098ec0 0x7f42c00afee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.130+0000 7f42c7e24700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42c0069f30 0x7f42c006a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.130+0000 7f42c7e24700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c006a910 con 0x7f42c015c740 2023-07-08T12:39:36.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.130+0000 7f42c7e24700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c00cea70 con 0x7f42c0098ec0 2023-07-08T12:39:36.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.130+0000 7f42c7e24700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c00b0850 con 0x7f42c0069f30 2023-07-08T12:39:36.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c5bc0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f42c015c740 0x7f42c00cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c5bc0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f42c015c740 0x7f42c00cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51828/0 (socket says 172.21.15.12:51828) 2023-07-08T12:39:36.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c5bc0700 1 -- 172.21.15.12:0/531763873 learned_addr learned my addr 172.21.15.12:0/531763873 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:36.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c53bf700 1 --2- 172.21.15.12:0/531763873 >> v2:172.21.15.160:3300/0 conn(0x7f42c0098ec0 0x7f42c00afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c63c1700 1 --2- 172.21.15.12:0/531763873 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42c0069f30 0x7f42c006a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c5bc0700 1 -- 172.21.15.12:0/531763873 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42c0069f30 msgr2=0x7f42c006a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:36.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c5bc0700 1 --2- 172.21.15.12:0/531763873 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42c0069f30 0x7f42c006a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c5bc0700 1 -- 172.21.15.12:0/531763873 >> v2:172.21.15.160:3300/0 conn(0x7f42c0098ec0 msgr2=0x7f42c00afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:36.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c5bc0700 1 --2- 172.21.15.12:0/531763873 >> v2:172.21.15.160:3300/0 conn(0x7f42c0098ec0 0x7f42c00afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c5bc0700 1 -- 172.21.15.12:0/531763873 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c00ce420 con 0x7f42c015c740 2023-07-08T12:39:36.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c53bf700 1 --2- 172.21.15.12:0/531763873 >> v2:172.21.15.160:3300/0 conn(0x7f42c0098ec0 0x7f42c00afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:36.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c5bc0700 1 --2- 172.21.15.12:0/531763873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c015c740 0x7f42c00cef00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f42c0070690 tx=0x7f42bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb004a6cb42f1e1c server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:36.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c4bbe700 1 -- 172.21.15.12:0/531763873 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc00b870 con 0x7f42c015c740 2023-07-08T12:39:36.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c4bbe700 1 -- 172.21.15.12:0/531763873 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc00b9d0 con 0x7f42c015c740 2023-07-08T12:39:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.131+0000 7f42c4bbe700 1 -- 172.21.15.12:0/531763873 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc00bd60 con 0x7f42c015c740 2023-07-08T12:39:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.132+0000 7f42c7e24700 1 -- 172.21.15.12:0/531763873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c015c740 msgr2=0x7f42c00cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.132+0000 7f42c7e24700 1 --2- 172.21.15.12:0/531763873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c015c740 0x7f42c00cef00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f42c0070690 tx=0x7f42bc009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.132+0000 7f42c7e24700 1 -- 172.21.15.12:0/531763873 shutdown_connections 2023-07-08T12:39:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.132+0000 7f42c7e24700 1 --2- 172.21.15.12:0/531763873 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42c0069f30 0x7f42c006a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.132+0000 7f42c7e24700 1 --2- 172.21.15.12:0/531763873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c015c740 0x7f42c00cef00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.132+0000 7f42c7e24700 1 --2- 172.21.15.12:0/531763873 >> v2:172.21.15.160:3300/0 conn(0x7f42c0098ec0 0x7f42c00afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.132+0000 7f42c7e24700 1 -- 172.21.15.12:0/531763873 >> 172.21.15.12:0/531763873 conn(0x7f42c0098a30 msgr2=0x7f42c00cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:36.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.132+0000 7f42c7e24700 1 -- 172.21.15.12:0/531763873 shutdown_connections 2023-07-08T12:39:36.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.132+0000 7f42c7e24700 1 -- 172.21.15.12:0/531763873 wait complete. 2023-07-08T12:39:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.133+0000 7f42c7e24700 1 Processor -- start 2023-07-08T12:39:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.133+0000 7f42c7e24700 1 -- start start 2023-07-08T12:39:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c7e24700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c0069f30 0x7f42c006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c7e24700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42c0098ec0 0x7f42c01f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c7e24700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42c015c740 0x7f42c00b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c7e24700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c006ea30 con 0x7f42c0069f30 2023-07-08T12:39:36.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c7e24700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c006de20 con 0x7f42c0098ec0 2023-07-08T12:39:36.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c7e24700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c006e100 con 0x7f42c015c740 2023-07-08T12:39:36.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c5bc0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c0069f30 0x7f42c006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c5bc0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c0069f30 0x7f42c006d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51834/0 (socket says 172.21.15.12:51834) 2023-07-08T12:39:36.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c5bc0700 1 -- 172.21.15.12:0/2171898867 learned_addr learned my addr 172.21.15.12:0/2171898867 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:36.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c53bf700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42c0098ec0 0x7f42c01f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c63c1700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42c015c740 0x7f42c00b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c5bc0700 1 -- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42c015c740 msgr2=0x7f42c00b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c5bc0700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42c015c740 0x7f42c00b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c5bc0700 1 -- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42c0098ec0 msgr2=0x7f42c01f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c5bc0700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42c0098ec0 0x7f42c01f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c5bc0700 1 -- 172.21.15.12:0/2171898867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc009550 con 0x7f42c0069f30 2023-07-08T12:39:36.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.134+0000 7f42c5bc0700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c0069f30 0x7f42c006d5f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009c10 tx=0x7f42bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:36.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.135+0000 7f42b27fc700 1 -- 172.21.15.12:0/2171898867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc004250 con 0x7f42c0069f30 2023-07-08T12:39:36.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.135+0000 7f42b27fc700 1 -- 172.21.15.12:0/2171898867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc01f070 con 0x7f42c0069f30 2023-07-08T12:39:36.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.135+0000 7f42c7e24700 1 -- 172.21.15.12:0/2171898867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c006e3e0 con 0x7f42c0069f30 2023-07-08T12:39:36.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.135+0000 7f42b27fc700 1 -- 172.21.15.12:0/2171898867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0045d0 con 0x7f42c0069f30 2023-07-08T12:39:36.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.135+0000 7f42c7e24700 1 -- 172.21.15.12:0/2171898867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c006db30 con 0x7f42c0069f30 2023-07-08T12:39:36.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.135+0000 7f42c7e24700 1 -- 172.21.15.12:0/2171898867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c00cde60 con 0x7f42c0069f30 2023-07-08T12:39:36.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.136+0000 7f42b27fc700 1 -- 172.21.15.12:0/2171898867 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f42bc021030 con 0x7f42c0069f30 2023-07-08T12:39:36.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.136+0000 7f42b27fc700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f42a406ace0 0x7f42a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.136+0000 7f42b27fc700 1 -- 172.21.15.12:0/2171898867 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f42bc08a8d0 con 0x7f42c0069f30 2023-07-08T12:39:36.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.137+0000 7f42c53bf700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f42a406ace0 0x7f42a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.137+0000 7f42c53bf700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f42a406ace0 0x7f42a406d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f42c01495c0 tx=0x7f42b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:36.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.141+0000 7f42b27fc700 1 -- 172.21.15.12:0/2171898867 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f42bc055970 con 0x7f42c0069f30 2023-07-08T12:39:36.192 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:39:36.195 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:39:36.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.335+0000 7f42c7e24700 1 -- 172.21.15.12:0/2171898867 --> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42c00ce700 con 0x7f42a406ace0 2023-07-08T12:39:36.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.344+0000 7f42b27fc700 1 -- 172.21.15.12:0/2171898867 <== mgr.4103 v2:172.21.15.160:6832/550875184 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311372 (secure 0 0 0) 0x7f42b4011720 con 0x7f42a406ace0 2023-07-08T12:39:36.349 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:36.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.347+0000 7f42c7e24700 1 -- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f42a406ace0 msgr2=0x7f42a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.347+0000 7f42c7e24700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f42a406ace0 0x7f42a406d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f42c01495c0 tx=0x7f42b4008040 comp rx=0 tx=0).stop 2023-07-08T12:39:36.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.347+0000 7f42c7e24700 1 -- 172.21.15.12:0/2171898867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c0069f30 msgr2=0x7f42c006d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.347+0000 7f42c7e24700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c0069f30 0x7f42c006d5f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009c10 tx=0x7f42bc004750 comp rx=0 tx=0).stop 2023-07-08T12:39:36.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.347+0000 7f42c7e24700 1 -- 172.21.15.12:0/2171898867 shutdown_connections 2023-07-08T12:39:36.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.347+0000 7f42c7e24700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f42a406ace0 0x7f42a406d190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.347+0000 7f42c7e24700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42c015c740 0x7f42c00b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.347+0000 7f42c7e24700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42c0069f30 0x7f42c006d5f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.347+0000 7f42c7e24700 1 --2- 172.21.15.12:0/2171898867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42c0098ec0 0x7f42c01f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.347+0000 7f42c7e24700 1 -- 172.21.15.12:0/2171898867 >> 172.21.15.12:0/2171898867 conn(0x7f42c0098a30 msgr2=0x7f42c00af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:36.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.347+0000 7f42c7e24700 1 -- 172.21.15.12:0/2171898867 shutdown_connections 2023-07-08T12:39:36.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.347+0000 7f42c7e24700 1 -- 172.21.15.12:0/2171898867 wait complete. 2023-07-08T12:39:36.353 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-07-08T12:39:36.362 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2023-07-08T12:39:36.163036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220636,"kb_used_data":5020,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754084,"statfs":{"total":773094113280,"available":772868182016,"internally_reserved":0,"allocated":5140480,"data_stored":2918095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001345"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823180+0000","last_change":"2023-07-08T12:39:31.840762+0000","last_active":"2023-07-08T12:39:33.823180+0000","last_peered":"2023-07-08T12:39:33.823180+0000","last_clean":"2023-07-08T12:39:33.823180+0000","last_became_active":"2023-07-08T12:39:31.840502+0000","last_became_peered":"2023-07-08T12:39:31.840502+0000","last_unstale":"2023-07-08T12:39:33.823180+0000","last_undegraded":"2023-07-08T12:39:33.823180+0000","last_fullsized":"2023-07-08T12:39:33.823180+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:40:16.735216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.833208+0000","last_change":"2023-07-08T12:39:31.848138+0000","last_active":"2023-07-08T12:39:32.833208+0000","last_peered":"2023-07-08T12:39:32.833208+0000","last_clean":"2023-07-08T12:39:32.833208+0000","last_became_active":"2023-07-08T12:39:31.847865+0000","last_became_peered":"2023-07-08T12:39:31.847865+0000","last_unstale":"2023-07-08T12:39:32.833208+0000","last_undegraded":"2023-07-08T12:39:32.833208+0000","last_fullsized":"2023-07-08T12:39:32.833208+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:17:47.026471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823871+0000","last_change":"2023-07-08T12:39:31.846667+0000","last_active":"2023-07-08T12:39:33.823871+0000","last_peered":"2023-07-08T12:39:33.823871+0000","last_clean":"2023-07-08T12:39:33.823871+0000","last_became_active":"2023-07-08T12:39:31.846423+0000","last_became_peered":"2023-07-08T12:39:31.846423+0000","last_unstale":"2023-07-08T12:39:33.823871+0000","last_undegraded":"2023-07-08T12:39:33.823871+0000","last_fullsized":"2023-07-08T12:39:33.823871+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:14:42.560815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823748+0000","last_change":"2023-07-08T12:39:31.843550+0000","last_active":"2023-07-08T12:39:33.823748+0000","last_peered":"2023-07-08T12:39:33.823748+0000","last_clean":"2023-07-08T12:39:33.823748+0000","last_became_active":"2023-07-08T12:39:31.843259+0000","last_became_peered":"2023-07-08T12:39:31.843259+0000","last_unstale":"2023-07-08T12:39:33.823748+0000","last_undegraded":"2023-07-08T12:39:33.823748+0000","last_fullsized":"2023-07-08T12:39:33.823748+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:19:47.454309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832918+0000","last_change":"2023-07-08T12:39:31.848085+0000","last_active":"2023-07-08T12:39:32.832918+0000","last_peered":"2023-07-08T12:39:32.832918+0000","last_clean":"2023-07-08T12:39:32.832918+0000","last_became_active":"2023-07-08T12:39:31.847794+0000","last_became_peered":"2023-07-08T12:39:31.847794+0000","last_unstale":"2023-07-08T12:39:32.832918+0000","last_undegraded":"2023-07-08T12:39:32.832918+0000","last_fullsized":"2023-07-08T12:39:32.832918+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:43:40.618806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823865+0000","last_change":"2023-07-08T12:39:31.831299+0000","last_active":"2023-07-08T12:39:33.823865+0000","last_peered":"2023-07-08T12:39:33.823865+0000","last_clean":"2023-07-08T12:39:33.823865+0000","last_became_active":"2023-07-08T12:39:31.830986+0000","last_became_peered":"2023-07-08T12:39:31.830986+0000","last_unstale":"2023-07-08T12:39:33.823865+0000","last_undegraded":"2023-07-08T12:39:33.823865+0000","last_fullsized":"2023-07-08T12:39:33.823865+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:25:10.705665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824313+0000","last_change":"2023-07-08T12:39:31.837251+0000","last_active":"2023-07-08T12:39:33.824313+0000","last_peered":"2023-07-08T12:39:33.824313+0000","last_clean":"2023-07-08T12:39:33.824313+0000","last_became_active":"2023-07-08T12:39:31.836935+0000","last_became_peered":"2023-07-08T12:39:31.836935+0000","last_unstale":"2023-07-08T12:39:33.824313+0000","last_undegraded":"2023-07-08T12:39:33.824313+0000","last_fullsized":"2023-07-08T12:39:33.824313+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:35:09.187881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.822805+0000","last_change":"2023-07-08T12:39:31.854407+0000","last_active":"2023-07-08T12:39:33.822805+0000","last_peered":"2023-07-08T12:39:33.822805+0000","last_clean":"2023-07-08T12:39:33.822805+0000","last_became_active":"2023-07-08T12:39:31.853814+0000","last_became_peered":"2023-07-08T12:39:31.853814+0000","last_unstale":"2023-07-08T12:39:33.822805+0000","last_undegraded":"2023-07-08T12:39:33.822805+0000","last_fullsized":"2023-07-08T12:39:33.822805+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:39:33.754104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823584+0000","last_change":"2023-07-08T12:39:31.843710+0000","last_active":"2023-07-08T12:39:33.823584+0000","last_peered":"2023-07-08T12:39:33.823584+0000","last_clean":"2023-07-08T12:39:33.823584+0000","last_became_active":"2023-07-08T12:39:31.843446+0000","last_became_peered":"2023-07-08T12:39:31.843446+0000","last_unstale":"2023-07-08T12:39:33.823584+0000","last_undegraded":"2023-07-08T12:39:33.823584+0000","last_fullsized":"2023-07-08T12:39:33.823584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:08:14.092534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823722+0000","last_change":"2023-07-08T12:39:31.848190+0000","last_active":"2023-07-08T12:39:33.823722+0000","last_peered":"2023-07-08T12:39:33.823722+0000","last_clean":"2023-07-08T12:39:33.823722+0000","last_became_active":"2023-07-08T12:39:31.847587+0000","last_became_peered":"2023-07-08T12:39:31.847587+0000","last_unstale":"2023-07-08T12:39:33.823722+0000","last_undegraded":"2023-07-08T12:39:33.823722+0000","last_fullsized":"2023-07-08T12:39:33.823722+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:40:42.000948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823278+0000","last_change":"2023-07-08T12:39:31.855767+0000","last_active":"2023-07-08T12:39:33.823278+0000","last_peered":"2023-07-08T12:39:33.823278+0000","last_clean":"2023-07-08T12:39:33.823278+0000","last_became_active":"2023-07-08T12:39:31.855479+0000","last_became_peered":"2023-07-08T12:39:31.855479+0000","last_unstale":"2023-07-08T12:39:33.823278+0000","last_undegraded":"2023-07-08T12:39:33.823278+0000","last_fullsized":"2023-07-08T12:39:33.823278+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:37:58.717995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.833088+0000","last_change":"2023-07-08T12:39:31.846546+0000","last_active":"2023-07-08T12:39:32.833088+0000","last_peered":"2023-07-08T12:39:32.833088+0000","last_clean":"2023-07-08T12:39:32.833088+0000","last_became_active":"2023-07-08T12:39:31.846254+0000","last_became_peered":"2023-07-08T12:39:31.846254+0000","last_unstale":"2023-07-08T12:39:32.833088+0000","last_undegraded":"2023-07-08T12:39:32.833088+0000","last_fullsized":"2023-07-08T12:39:32.833088+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T16:57:22.673456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832401+0000","last_change":"2023-07-08T12:39:31.852950+0000","last_active":"2023-07-08T12:39:32.832401+0000","last_peered":"2023-07-08T12:39:32.832401+0000","last_clean":"2023-07-08T12:39:32.832401+0000","last_became_active":"2023-07-08T12:39:31.852662+0000","last_became_peered":"2023-07-08T12:39:31.852662+0000","last_unstale":"2023-07-08T12:39:32.832401+0000","last_undegraded":"2023-07-08T12:39:32.832401+0000","last_fullsized":"2023-07-08T12:39:32.832401+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:29:33.812895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.822921+0000","last_change":"2023-07-08T12:39:31.844627+0000","last_active":"2023-07-08T12:39:33.822921+0000","last_peered":"2023-07-08T12:39:33.822921+0000","last_clean":"2023-07-08T12:39:33.822921+0000","last_became_active":"2023-07-08T12:39:31.844125+0000","last_became_peered":"2023-07-08T12:39:31.844125+0000","last_unstale":"2023-07-08T12:39:33.822921+0000","last_undegraded":"2023-07-08T12:39:33.822921+0000","last_fullsized":"2023-07-08T12:39:33.822921+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:13:37.189999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824209+0000","last_change":"2023-07-08T12:39:31.839136+0000","last_active":"2023-07-08T12:39:33.824209+0000","last_peered":"2023-07-08T12:39:33.824209+0000","last_clean":"2023-07-08T12:39:33.824209+0000","last_became_active":"2023-07-08T12:39:31.838857+0000","last_became_peered":"2023-07-08T12:39:31.838857+0000","last_unstale":"2023-07-08T12:39:33.824209+0000","last_undegraded":"2023-07-08T12:39:33.824209+0000","last_fullsized":"2023-07-08T12:39:33.824209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:44:53.402895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823074+0000","last_change":"2023-07-08T12:39:31.856141+0000","last_active":"2023-07-08T12:39:33.823074+0000","last_peered":"2023-07-08T12:39:33.823074+0000","last_clean":"2023-07-08T12:39:33.823074+0000","last_became_active":"2023-07-08T12:39:31.855823+0000","last_became_peered":"2023-07-08T12:39:31.855823+0000","last_unstale":"2023-07-08T12:39:33.823074+0000","last_undegraded":"2023-07-08T12:39:33.823074+0000","last_fullsized":"2023-07-08T12:39:33.823074+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:30:30.280667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824689+0000","last_change":"2023-07-08T12:39:31.842173+0000","last_active":"2023-07-08T12:39:33.824689+0000","last_peered":"2023-07-08T12:39:33.824689+0000","last_clean":"2023-07-08T12:39:33.824689+0000","last_became_active":"2023-07-08T12:39:31.841919+0000","last_became_peered":"2023-07-08T12:39:31.841919+0000","last_unstale":"2023-07-08T12:39:33.824689+0000","last_undegraded":"2023-07-08T12:39:33.824689+0000","last_fullsized":"2023-07-08T12:39:33.824689+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:22:04.307511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824594+0000","last_change":"2023-07-08T12:39:31.856423+0000","last_active":"2023-07-08T12:39:33.824594+0000","last_peered":"2023-07-08T12:39:33.824594+0000","last_clean":"2023-07-08T12:39:33.824594+0000","last_became_active":"2023-07-08T12:39:31.856139+0000","last_became_peered":"2023-07-08T12:39:31.856139+0000","last_unstale":"2023-07-08T12:39:33.824594+0000","last_undegraded":"2023-07-08T12:39:33.824594+0000","last_fullsized":"2023-07-08T12:39:33.824594+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:37:57.072499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824114+0000","last_change":"2023-07-08T12:39:31.835930+0000","last_active":"2023-07-08T12:39:33.824114+0000","last_peered":"2023-07-08T12:39:33.824114+0000","last_clean":"2023-07-08T12:39:33.824114+0000","last_became_active":"2023-07-08T12:39:31.835610+0000","last_became_peered":"2023-07-08T12:39:31.835610+0000","last_unstale":"2023-07-08T12:39:33.824114+0000","last_undegraded":"2023-07-08T12:39:33.824114+0000","last_fullsized":"2023-07-08T12:39:33.824114+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:16:16.262329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832124+0000","last_change":"2023-07-08T12:39:31.848779+0000","last_active":"2023-07-08T12:39:32.832124+0000","last_peered":"2023-07-08T12:39:32.832124+0000","last_clean":"2023-07-08T12:39:32.832124+0000","last_became_active":"2023-07-08T12:39:31.848523+0000","last_became_peered":"2023-07-08T12:39:31.848523+0000","last_unstale":"2023-07-08T12:39:32.832124+0000","last_undegraded":"2023-07-08T12:39:32.832124+0000","last_fullsized":"2023-07-08T12:39:32.832124+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:51:03.813375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823522+0000","last_change":"2023-07-08T12:39:31.836957+0000","last_active":"2023-07-08T12:39:33.823522+0000","last_peered":"2023-07-08T12:39:33.823522+0000","last_clean":"2023-07-08T12:39:33.823522+0000","last_became_active":"2023-07-08T12:39:31.836655+0000","last_became_peered":"2023-07-08T12:39:31.836655+0000","last_unstale":"2023-07-08T12:39:33.823522+0000","last_undegraded":"2023-07-08T12:39:33.823522+0000","last_fullsized":"2023-07-08T12:39:33.823522+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:44:13.834042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823112+0000","last_change":"2023-07-08T12:39:31.844441+0000","last_active":"2023-07-08T12:39:33.823112+0000","last_peered":"2023-07-08T12:39:33.823112+0000","last_clean":"2023-07-08T12:39:33.823112+0000","last_became_active":"2023-07-08T12:39:31.842903+0000","last_became_peered":"2023-07-08T12:39:31.842903+0000","last_unstale":"2023-07-08T12:39:33.823112+0000","last_undegraded":"2023-07-08T12:39:33.823112+0000","last_fullsized":"2023-07-08T12:39:33.823112+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:47:55.427272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.822983+0000","last_change":"2023-07-08T12:39:31.832233+0000","last_active":"2023-07-08T12:39:33.822983+0000","last_peered":"2023-07-08T12:39:33.822983+0000","last_clean":"2023-07-08T12:39:33.822983+0000","last_became_active":"2023-07-08T12:39:31.831618+0000","last_became_peered":"2023-07-08T12:39:31.831618+0000","last_unstale":"2023-07-08T12:39:33.822983+0000","last_undegraded":"2023-07-08T12:39:33.822983+0000","last_fullsized":"2023-07-08T12:39:33.822983+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:29:33.812895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823238+0000","last_change":"2023-07-08T12:39:31.834083+0000","last_active":"2023-07-08T12:39:33.823238+0000","last_peered":"2023-07-08T12:39:33.823238+0000","last_clean":"2023-07-08T12:39:33.823238+0000","last_became_active":"2023-07-08T12:39:31.833850+0000","last_became_peered":"2023-07-08T12:39:31.833850+0000","last_unstale":"2023-07-08T12:39:33.823238+0000","last_undegraded":"2023-07-08T12:39:33.823238+0000","last_fullsized":"2023-07-08T12:39:33.823238+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T16:02:03.369571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824493+0000","last_change":"2023-07-08T12:39:31.837484+0000","last_active":"2023-07-08T12:39:33.824493+0000","last_peered":"2023-07-08T12:39:33.824493+0000","last_clean":"2023-07-08T12:39:33.824493+0000","last_became_active":"2023-07-08T12:39:31.837215+0000","last_became_peered":"2023-07-08T12:39:31.837215+0000","last_unstale":"2023-07-08T12:39:33.824493+0000","last_undegraded":"2023-07-08T12:39:33.824493+0000","last_fullsized":"2023-07-08T12:39:33.824493+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:29:18.797123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832952+0000","last_change":"2023-07-08T12:39:31.845453+0000","last_active":"2023-07-08T12:39:32.832952+0000","last_peered":"2023-07-08T12:39:32.832952+0000","last_clean":"2023-07-08T12:39:32.832952+0000","last_became_active":"2023-07-08T12:39:31.844612+0000","last_became_peered":"2023-07-08T12:39:31.844612+0000","last_unstale":"2023-07-08T12:39:32.832952+0000","last_undegraded":"2023-07-08T12:39:32.832952+0000","last_fullsized":"2023-07-08T12:39:32.832952+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:36:38.481738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824088+0000","last_change":"2023-07-08T12:39:31.842132+0000","last_active":"2023-07-08T12:39:33.824088+0000","last_peered":"2023-07-08T12:39:33.824088+0000","last_clean":"2023-07-08T12:39:33.824088+0000","last_became_active":"2023-07-08T12:39:31.840141+0000","last_became_peered":"2023-07-08T12:39:31.840141+0000","last_unstale":"2023-07-08T12:39:33.824088+0000","last_undegraded":"2023-07-08T12:39:33.824088+0000","last_fullsized":"2023-07-08T12:39:33.824088+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:17:31.626783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832844+0000","last_change":"2023-07-08T12:39:31.844835+0000","last_active":"2023-07-08T12:39:32.832844+0000","last_peered":"2023-07-08T12:39:32.832844+0000","last_clean":"2023-07-08T12:39:32.832844+0000","last_became_active":"2023-07-08T12:39:31.844543+0000","last_became_peered":"2023-07-08T12:39:31.844543+0000","last_unstale":"2023-07-08T12:39:32.832844+0000","last_undegraded":"2023-07-08T12:39:32.832844+0000","last_fullsized":"2023-07-08T12:39:32.832844+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:37.647056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824355+0000","last_change":"2023-07-08T12:39:31.838154+0000","last_active":"2023-07-08T12:39:33.824355+0000","last_peered":"2023-07-08T12:39:33.824355+0000","last_clean":"2023-07-08T12:39:33.824355+0000","last_became_active":"2023-07-08T12:39:31.837861+0000","last_became_peered":"2023-07-08T12:39:31.837861+0000","last_unstale":"2023-07-08T12:39:33.824355+0000","last_undegraded":"2023-07-08T12:39:33.824355+0000","last_fullsized":"2023-07-08T12:39:33.824355+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:33:15.950911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824007+0000","last_change":"2023-07-08T12:39:31.839860+0000","last_active":"2023-07-08T12:39:33.824007+0000","last_peered":"2023-07-08T12:39:33.824007+0000","last_clean":"2023-07-08T12:39:33.824007+0000","last_became_active":"2023-07-08T12:39:31.839556+0000","last_became_peered":"2023-07-08T12:39:31.839556+0000","last_unstale":"2023-07-08T12:39:33.824007+0000","last_undegraded":"2023-07-08T12:39:33.824007+0000","last_fullsized":"2023-07-08T12:39:33.824007+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:40:22.683204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823024+0000","last_change":"2023-07-08T12:39:31.835182+0000","last_active":"2023-07-08T12:39:33.823024+0000","last_peered":"2023-07-08T12:39:33.823024+0000","last_clean":"2023-07-08T12:39:33.823024+0000","last_became_active":"2023-07-08T12:39:31.831916+0000","last_became_peered":"2023-07-08T12:39:31.831916+0000","last_unstale":"2023-07-08T12:39:33.823024+0000","last_undegraded":"2023-07-08T12:39:33.823024+0000","last_fullsized":"2023-07-08T12:39:33.823024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:19:47.454309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.831908+0000","last_change":"2023-07-08T12:39:31.839674+0000","last_active":"2023-07-08T12:39:32.831908+0000","last_peered":"2023-07-08T12:39:32.831908+0000","last_clean":"2023-07-08T12:39:32.831908+0000","last_became_active":"2023-07-08T12:39:31.839197+0000","last_became_peered":"2023-07-08T12:39:31.839197+0000","last_unstale":"2023-07-08T12:39:32.831908+0000","last_undegraded":"2023-07-08T12:39:32.831908+0000","last_fullsized":"2023-07-08T12:39:32.831908+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:39:58.611275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832741+0000","last_change":"2023-07-08T12:39:31.846739+0000","last_active":"2023-07-08T12:39:32.832741+0000","last_peered":"2023-07-08T12:39:32.832741+0000","last_clean":"2023-07-08T12:39:32.832741+0000","last_became_active":"2023-07-08T12:39:31.846424+0000","last_became_peered":"2023-07-08T12:39:31.846424+0000","last_unstale":"2023-07-08T12:39:32.832741+0000","last_undegraded":"2023-07-08T12:39:32.832741+0000","last_fullsized":"2023-07-08T12:39:32.832741+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:46:01.949692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"41'4","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824027+0000","last_change":"2023-07-08T12:39:30.812603+0000","last_active":"2023-07-08T12:39:33.824027+0000","last_peered":"2023-07-08T12:39:33.824027+0000","last_clean":"2023-07-08T12:39:33.824027+0000","last_became_active":"2023-07-08T12:39:30.812263+0000","last_became_peered":"2023-07-08T12:39:30.812263+0000","last_unstale":"2023-07-08T12:39:33.824027+0000","last_undegraded":"2023-07-08T12:39:33.824027+0000","last_fullsized":"2023-07-08T12:39:33.824027+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:56:29.240435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824157+0000","last_change":"2023-07-08T12:39:31.838438+0000","last_active":"2023-07-08T12:39:33.824157+0000","last_peered":"2023-07-08T12:39:33.824157+0000","last_clean":"2023-07-08T12:39:33.824157+0000","last_became_active":"2023-07-08T12:39:31.838069+0000","last_became_peered":"2023-07-08T12:39:31.838069+0000","last_unstale":"2023-07-08T12:39:33.824157+0000","last_undegraded":"2023-07-08T12:39:33.824157+0000","last_fullsized":"2023-07-08T12:39:33.824157+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:10:44.904269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824872+0000","last_change":"2023-07-08T12:39:30.811823+0000","last_active":"2023-07-08T12:39:33.824872+0000","last_peered":"2023-07-08T12:39:33.824872+0000","last_clean":"2023-07-08T12:39:33.824872+0000","last_became_active":"2023-07-08T12:39:30.811563+0000","last_became_peered":"2023-07-08T12:39:30.811563+0000","last_unstale":"2023-07-08T12:39:33.824872+0000","last_undegraded":"2023-07-08T12:39:33.824872+0000","last_fullsized":"2023-07-08T12:39:33.824872+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T12:46:37.594400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832907+0000","last_change":"2023-07-08T12:39:31.843657+0000","last_active":"2023-07-08T12:39:32.832907+0000","last_peered":"2023-07-08T12:39:32.832907+0000","last_clean":"2023-07-08T12:39:32.832907+0000","last_became_active":"2023-07-08T12:39:31.843385+0000","last_became_peered":"2023-07-08T12:39:31.843385+0000","last_unstale":"2023-07-08T12:39:32.832907+0000","last_undegraded":"2023-07-08T12:39:32.832907+0000","last_fullsized":"2023-07-08T12:39:32.832907+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:01:22.029795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832873+0000","last_change":"2023-07-08T12:39:30.805977+0000","last_active":"2023-07-08T12:39:32.832873+0000","last_peered":"2023-07-08T12:39:32.832873+0000","last_clean":"2023-07-08T12:39:32.832873+0000","last_became_active":"2023-07-08T12:39:30.805690+0000","last_became_peered":"2023-07-08T12:39:30.805690+0000","last_unstale":"2023-07-08T12:39:32.832873+0000","last_undegraded":"2023-07-08T12:39:32.832873+0000","last_fullsized":"2023-07-08T12:39:32.832873+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:38:29.946772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832763+0000","last_change":"2023-07-08T12:39:31.837960+0000","last_active":"2023-07-08T12:39:32.832763+0000","last_peered":"2023-07-08T12:39:32.832763+0000","last_clean":"2023-07-08T12:39:32.832763+0000","last_became_active":"2023-07-08T12:39:31.837639+0000","last_became_peered":"2023-07-08T12:39:31.837639+0000","last_unstale":"2023-07-08T12:39:32.832763+0000","last_undegraded":"2023-07-08T12:39:32.832763+0000","last_fullsized":"2023-07-08T12:39:32.832763+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:27:58.794206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823133+0000","last_change":"2023-07-08T12:39:30.807411+0000","last_active":"2023-07-08T12:39:33.823133+0000","last_peered":"2023-07-08T12:39:33.823133+0000","last_clean":"2023-07-08T12:39:33.823133+0000","last_became_active":"2023-07-08T12:39:30.807104+0000","last_became_peered":"2023-07-08T12:39:30.807104+0000","last_unstale":"2023-07-08T12:39:33.823133+0000","last_undegraded":"2023-07-08T12:39:33.823133+0000","last_fullsized":"2023-07-08T12:39:33.823133+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:36.645028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824160+0000","last_change":"2023-07-08T12:39:31.839271+0000","last_active":"2023-07-08T12:39:33.824160+0000","last_peered":"2023-07-08T12:39:33.824160+0000","last_clean":"2023-07-08T12:39:33.824160+0000","last_became_active":"2023-07-08T12:39:31.839009+0000","last_became_peered":"2023-07-08T12:39:31.839009+0000","last_unstale":"2023-07-08T12:39:33.824160+0000","last_undegraded":"2023-07-08T12:39:33.824160+0000","last_fullsized":"2023-07-08T12:39:33.824160+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:20:05.181397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823701+0000","last_change":"2023-07-08T12:39:30.806544+0000","last_active":"2023-07-08T12:39:33.823701+0000","last_peered":"2023-07-08T12:39:33.823701+0000","last_clean":"2023-07-08T12:39:33.823701+0000","last_became_active":"2023-07-08T12:39:30.805403+0000","last_became_peered":"2023-07-08T12:39:30.805403+0000","last_unstale":"2023-07-08T12:39:33.823701+0000","last_undegraded":"2023-07-08T12:39:33.823701+0000","last_fullsized":"2023-07-08T12:39:33.823701+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:18:38.536278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823036+0000","last_change":"2023-07-08T12:39:31.855789+0000","last_active":"2023-07-08T12:39:33.823036+0000","last_peered":"2023-07-08T12:39:33.823036+0000","last_clean":"2023-07-08T12:39:33.823036+0000","last_became_active":"2023-07-08T12:39:31.855495+0000","last_became_peered":"2023-07-08T12:39:31.855495+0000","last_unstale":"2023-07-08T12:39:33.823036+0000","last_undegraded":"2023-07-08T12:39:33.823036+0000","last_fullsized":"2023-07-08T12:39:33.823036+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:45:37.868397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824255+0000","last_change":"2023-07-08T12:39:30.794331+0000","last_active":"2023-07-08T12:39:33.824255+0000","last_peered":"2023-07-08T12:39:33.824255+0000","last_clean":"2023-07-08T12:39:33.824255+0000","last_became_active":"2023-07-08T12:39:30.794098+0000","last_became_peered":"2023-07-08T12:39:30.794098+0000","last_unstale":"2023-07-08T12:39:33.824255+0000","last_undegraded":"2023-07-08T12:39:33.824255+0000","last_fullsized":"2023-07-08T12:39:33.824255+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:02:12.928657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.822803+0000","last_change":"2023-07-08T12:39:31.854585+0000","last_active":"2023-07-08T12:39:33.822803+0000","last_peered":"2023-07-08T12:39:33.822803+0000","last_clean":"2023-07-08T12:39:33.822803+0000","last_became_active":"2023-07-08T12:39:31.854117+0000","last_became_peered":"2023-07-08T12:39:31.854117+0000","last_unstale":"2023-07-08T12:39:33.822803+0000","last_undegraded":"2023-07-08T12:39:33.822803+0000","last_fullsized":"2023-07-08T12:39:33.822803+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:37:29.911697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832656+0000","last_change":"2023-07-08T12:39:30.804332+0000","last_active":"2023-07-08T12:39:32.832656+0000","last_peered":"2023-07-08T12:39:32.832656+0000","last_clean":"2023-07-08T12:39:32.832656+0000","last_became_active":"2023-07-08T12:39:30.804039+0000","last_became_peered":"2023-07-08T12:39:30.804039+0000","last_unstale":"2023-07-08T12:39:32.832656+0000","last_undegraded":"2023-07-08T12:39:32.832656+0000","last_fullsized":"2023-07-08T12:39:32.832656+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:45:17.049244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823098+0000","last_change":"2023-07-08T12:39:31.832404+0000","last_active":"2023-07-08T12:39:33.823098+0000","last_peered":"2023-07-08T12:39:33.823098+0000","last_clean":"2023-07-08T12:39:33.823098+0000","last_became_active":"2023-07-08T12:39:31.831963+0000","last_became_peered":"2023-07-08T12:39:31.831963+0000","last_unstale":"2023-07-08T12:39:33.823098+0000","last_undegraded":"2023-07-08T12:39:33.823098+0000","last_fullsized":"2023-07-08T12:39:33.823098+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:36:38.481738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823964+0000","last_change":"2023-07-08T12:39:30.803813+0000","last_active":"2023-07-08T12:39:33.823964+0000","last_peered":"2023-07-08T12:39:33.823964+0000","last_clean":"2023-07-08T12:39:33.823964+0000","last_became_active":"2023-07-08T12:39:30.803476+0000","last_became_peered":"2023-07-08T12:39:30.803476+0000","last_unstale":"2023-07-08T12:39:33.823964+0000","last_undegraded":"2023-07-08T12:39:33.823964+0000","last_fullsized":"2023-07-08T12:39:33.823964+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:08:19.982607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824844+0000","last_change":"2023-07-08T12:39:31.856241+0000","last_active":"2023-07-08T12:39:33.824844+0000","last_peered":"2023-07-08T12:39:33.824844+0000","last_clean":"2023-07-08T12:39:33.824844+0000","last_became_active":"2023-07-08T12:39:31.855802+0000","last_became_peered":"2023-07-08T12:39:31.855802+0000","last_unstale":"2023-07-08T12:39:33.824844+0000","last_undegraded":"2023-07-08T12:39:33.824844+0000","last_fullsized":"2023-07-08T12:39:33.824844+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:57:57.633115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824838+0000","last_change":"2023-07-08T12:39:30.806827+0000","last_active":"2023-07-08T12:39:33.824838+0000","last_peered":"2023-07-08T12:39:33.824838+0000","last_clean":"2023-07-08T12:39:33.824838+0000","last_became_active":"2023-07-08T12:39:30.806549+0000","last_became_peered":"2023-07-08T12:39:30.806549+0000","last_unstale":"2023-07-08T12:39:33.824838+0000","last_undegraded":"2023-07-08T12:39:33.824838+0000","last_fullsized":"2023-07-08T12:39:33.824838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:48:00.768371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823410+0000","last_change":"2023-07-08T12:39:31.826411+0000","last_active":"2023-07-08T12:39:33.823410+0000","last_peered":"2023-07-08T12:39:33.823410+0000","last_clean":"2023-07-08T12:39:33.823410+0000","last_became_active":"2023-07-08T12:39:31.826097+0000","last_became_peered":"2023-07-08T12:39:31.826097+0000","last_unstale":"2023-07-08T12:39:33.823410+0000","last_undegraded":"2023-07-08T12:39:33.823410+0000","last_fullsized":"2023-07-08T12:39:33.823410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:43:40.618806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823844+0000","last_change":"2023-07-08T12:39:30.807687+0000","last_active":"2023-07-08T12:39:33.823844+0000","last_peered":"2023-07-08T12:39:33.823844+0000","last_clean":"2023-07-08T12:39:33.823844+0000","last_became_active":"2023-07-08T12:39:30.807348+0000","last_became_peered":"2023-07-08T12:39:30.807348+0000","last_unstale":"2023-07-08T12:39:33.823844+0000","last_undegraded":"2023-07-08T12:39:33.823844+0000","last_fullsized":"2023-07-08T12:39:33.823844+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:23:31.199220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823778+0000","last_change":"2023-07-08T12:39:31.846533+0000","last_active":"2023-07-08T12:39:33.823778+0000","last_peered":"2023-07-08T12:39:33.823778+0000","last_clean":"2023-07-08T12:39:33.823778+0000","last_became_active":"2023-07-08T12:39:31.846057+0000","last_became_peered":"2023-07-08T12:39:31.846057+0000","last_unstale":"2023-07-08T12:39:33.823778+0000","last_undegraded":"2023-07-08T12:39:33.823778+0000","last_fullsized":"2023-07-08T12:39:33.823778+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:57:49.872934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"41'3","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823660+0000","last_change":"2023-07-08T12:39:30.812604+0000","last_active":"2023-07-08T12:39:33.823660+0000","last_peered":"2023-07-08T12:39:33.823660+0000","last_clean":"2023-07-08T12:39:33.823660+0000","last_became_active":"2023-07-08T12:39:30.812305+0000","last_became_peered":"2023-07-08T12:39:30.812305+0000","last_unstale":"2023-07-08T12:39:33.823660+0000","last_undegraded":"2023-07-08T12:39:33.823660+0000","last_fullsized":"2023-07-08T12:39:33.823660+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:05:27.243194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824056+0000","last_change":"2023-07-08T12:39:31.839000+0000","last_active":"2023-07-08T12:39:33.824056+0000","last_peered":"2023-07-08T12:39:33.824056+0000","last_clean":"2023-07-08T12:39:33.824056+0000","last_became_active":"2023-07-08T12:39:31.838705+0000","last_became_peered":"2023-07-08T12:39:31.838705+0000","last_unstale":"2023-07-08T12:39:33.824056+0000","last_undegraded":"2023-07-08T12:39:33.824056+0000","last_fullsized":"2023-07-08T12:39:33.824056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:05:58.914346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"41'6","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832259+0000","last_change":"2023-07-08T12:39:30.808178+0000","last_active":"2023-07-08T12:39:32.832259+0000","last_peered":"2023-07-08T12:39:32.832259+0000","last_clean":"2023-07-08T12:39:32.832259+0000","last_became_active":"2023-07-08T12:39:30.807895+0000","last_became_peered":"2023-07-08T12:39:30.807895+0000","last_unstale":"2023-07-08T12:39:32.832259+0000","last_undegraded":"2023-07-08T12:39:32.832259+0000","last_fullsized":"2023-07-08T12:39:32.832259+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:17:08.777149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823607+0000","last_change":"2023-07-08T12:39:31.844505+0000","last_active":"2023-07-08T12:39:33.823607+0000","last_peered":"2023-07-08T12:39:33.823607+0000","last_clean":"2023-07-08T12:39:33.823607+0000","last_became_active":"2023-07-08T12:39:31.844143+0000","last_became_peered":"2023-07-08T12:39:31.844143+0000","last_unstale":"2023-07-08T12:39:33.823607+0000","last_undegraded":"2023-07-08T12:39:33.823607+0000","last_fullsized":"2023-07-08T12:39:33.823607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:00:38.651071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823879+0000","last_change":"2023-07-08T12:39:30.809206+0000","last_active":"2023-07-08T12:39:33.823879+0000","last_peered":"2023-07-08T12:39:33.823879+0000","last_clean":"2023-07-08T12:39:33.823879+0000","last_became_active":"2023-07-08T12:39:30.808804+0000","last_became_peered":"2023-07-08T12:39:30.808804+0000","last_unstale":"2023-07-08T12:39:33.823879+0000","last_undegraded":"2023-07-08T12:39:33.823879+0000","last_fullsized":"2023-07-08T12:39:33.823879+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T16:46:16.882778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823231+0000","last_change":"2023-07-08T12:39:31.844491+0000","last_active":"2023-07-08T12:39:33.823231+0000","last_peered":"2023-07-08T12:39:33.823231+0000","last_clean":"2023-07-08T12:39:33.823231+0000","last_became_active":"2023-07-08T12:39:31.844197+0000","last_became_peered":"2023-07-08T12:39:31.844197+0000","last_unstale":"2023-07-08T12:39:33.823231+0000","last_undegraded":"2023-07-08T12:39:33.823231+0000","last_fullsized":"2023-07-08T12:39:33.823231+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:40:01.451458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823347+0000","last_change":"2023-07-08T12:39:30.809322+0000","last_active":"2023-07-08T12:39:33.823347+0000","last_peered":"2023-07-08T12:39:33.823347+0000","last_clean":"2023-07-08T12:39:33.823347+0000","last_became_active":"2023-07-08T12:39:30.809045+0000","last_became_peered":"2023-07-08T12:39:30.809045+0000","last_unstale":"2023-07-08T12:39:33.823347+0000","last_undegraded":"2023-07-08T12:39:33.823347+0000","last_fullsized":"2023-07-08T12:39:33.823347+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:01:21.027767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823299+0000","last_change":"2023-07-08T12:39:31.836232+0000","last_active":"2023-07-08T12:39:33.823299+0000","last_peered":"2023-07-08T12:39:33.823299+0000","last_clean":"2023-07-08T12:39:33.823299+0000","last_became_active":"2023-07-08T12:39:31.835925+0000","last_became_peered":"2023-07-08T12:39:31.835925+0000","last_unstale":"2023-07-08T12:39:33.823299+0000","last_undegraded":"2023-07-08T12:39:33.823299+0000","last_fullsized":"2023-07-08T12:39:33.823299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:34:46.998464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823497+0000","last_change":"2023-07-08T12:39:30.806227+0000","last_active":"2023-07-08T12:39:33.823497+0000","last_peered":"2023-07-08T12:39:33.823497+0000","last_clean":"2023-07-08T12:39:33.823497+0000","last_became_active":"2023-07-08T12:39:30.805952+0000","last_became_peered":"2023-07-08T12:39:30.805952+0000","last_unstale":"2023-07-08T12:39:33.823497+0000","last_undegraded":"2023-07-08T12:39:33.823497+0000","last_fullsized":"2023-07-08T12:39:33.823497+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:08:19.982607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824182+0000","last_change":"2023-07-08T12:39:31.829836+0000","last_active":"2023-07-08T12:39:33.824182+0000","last_peered":"2023-07-08T12:39:33.824182+0000","last_clean":"2023-07-08T12:39:33.824182+0000","last_became_active":"2023-07-08T12:39:31.829533+0000","last_became_peered":"2023-07-08T12:39:31.829533+0000","last_unstale":"2023-07-08T12:39:33.824182+0000","last_undegraded":"2023-07-08T12:39:33.824182+0000","last_fullsized":"2023-07-08T12:39:33.824182+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:24:39.607051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"41'8","reported_seq":21,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.822962+0000","last_change":"2023-07-08T12:39:30.805316+0000","last_active":"2023-07-08T12:39:33.822962+0000","last_peered":"2023-07-08T12:39:33.822962+0000","last_clean":"2023-07-08T12:39:33.822962+0000","last_became_active":"2023-07-08T12:39:30.805004+0000","last_became_peered":"2023-07-08T12:39:30.805004+0000","last_unstale":"2023-07-08T12:39:33.822962+0000","last_undegraded":"2023-07-08T12:39:33.822962+0000","last_fullsized":"2023-07-08T12:39:33.822962+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:41:38.449779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832567+0000","last_change":"2023-07-08T12:39:31.841435+0000","last_active":"2023-07-08T12:39:32.832567+0000","last_peered":"2023-07-08T12:39:32.832567+0000","last_clean":"2023-07-08T12:39:32.832567+0000","last_became_active":"2023-07-08T12:39:31.833085+0000","last_became_peered":"2023-07-08T12:39:31.833085+0000","last_unstale":"2023-07-08T12:39:32.832567+0000","last_undegraded":"2023-07-08T12:39:32.832567+0000","last_fullsized":"2023-07-08T12:39:32.832567+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:33:15.950911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832340+0000","last_change":"2023-07-08T12:39:30.812384+0000","last_active":"2023-07-08T12:39:32.832340+0000","last_peered":"2023-07-08T12:39:32.832340+0000","last_clean":"2023-07-08T12:39:32.832340+0000","last_became_active":"2023-07-08T12:39:30.812035+0000","last_became_peered":"2023-07-08T12:39:30.812035+0000","last_unstale":"2023-07-08T12:39:32.832340+0000","last_undegraded":"2023-07-08T12:39:32.832340+0000","last_fullsized":"2023-07-08T12:39:32.832340+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:30:29.278638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823646+0000","last_change":"2023-07-08T12:39:31.851175+0000","last_active":"2023-07-08T12:39:33.823646+0000","last_peered":"2023-07-08T12:39:33.823646+0000","last_clean":"2023-07-08T12:39:33.823646+0000","last_became_active":"2023-07-08T12:39:31.848369+0000","last_became_peered":"2023-07-08T12:39:31.848369+0000","last_unstale":"2023-07-08T12:39:33.823646+0000","last_undegraded":"2023-07-08T12:39:33.823646+0000","last_fullsized":"2023-07-08T12:39:33.823646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:03:47.591545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824700+0000","last_change":"2023-07-08T12:39:30.803572+0000","last_active":"2023-07-08T12:39:33.824700+0000","last_peered":"2023-07-08T12:39:33.824700+0000","last_clean":"2023-07-08T12:39:33.824700+0000","last_became_active":"2023-07-08T12:39:30.803293+0000","last_became_peered":"2023-07-08T12:39:30.803293+0000","last_unstale":"2023-07-08T12:39:33.824700+0000","last_undegraded":"2023-07-08T12:39:33.824700+0000","last_fullsized":"2023-07-08T12:39:33.824700+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:59:56.705345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823190+0000","last_change":"2023-07-08T12:39:31.843661+0000","last_active":"2023-07-08T12:39:33.823190+0000","last_peered":"2023-07-08T12:39:33.823190+0000","last_clean":"2023-07-08T12:39:33.823190+0000","last_became_active":"2023-07-08T12:39:31.843401+0000","last_became_peered":"2023-07-08T12:39:31.843401+0000","last_unstale":"2023-07-08T12:39:33.823190+0000","last_undegraded":"2023-07-08T12:39:33.823190+0000","last_fullsized":"2023-07-08T12:39:33.823190+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:51:49.516096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.833140+0000","last_change":"2023-07-08T12:39:30.803369+0000","last_active":"2023-07-08T12:39:32.833140+0000","last_peered":"2023-07-08T12:39:32.833140+0000","last_clean":"2023-07-08T12:39:32.833140+0000","last_became_active":"2023-07-08T12:39:30.803115+0000","last_became_peered":"2023-07-08T12:39:30.803115+0000","last_unstale":"2023-07-08T12:39:32.833140+0000","last_undegraded":"2023-07-08T12:39:32.833140+0000","last_fullsized":"2023-07-08T12:39:32.833140+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:59:20.157151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824542+0000","last_change":"2023-07-08T12:39:31.839148+0000","last_active":"2023-07-08T12:39:33.824542+0000","last_peered":"2023-07-08T12:39:33.824542+0000","last_clean":"2023-07-08T12:39:33.824542+0000","last_became_active":"2023-07-08T12:39:31.838756+0000","last_became_peered":"2023-07-08T12:39:31.838756+0000","last_unstale":"2023-07-08T12:39:33.824542+0000","last_undegraded":"2023-07-08T12:39:33.824542+0000","last_fullsized":"2023-07-08T12:39:33.824542+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:25:46.408715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824024+0000","last_change":"2023-07-08T12:39:30.812807+0000","last_active":"2023-07-08T12:39:33.824024+0000","last_peered":"2023-07-08T12:39:33.824024+0000","last_clean":"2023-07-08T12:39:33.824024+0000","last_became_active":"2023-07-08T12:39:30.812282+0000","last_became_peered":"2023-07-08T12:39:30.812282+0000","last_unstale":"2023-07-08T12:39:33.824024+0000","last_undegraded":"2023-07-08T12:39:33.824024+0000","last_fullsized":"2023-07-08T12:39:33.824024+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:33:31.686095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823275+0000","last_change":"2023-07-08T12:39:31.835705+0000","last_active":"2023-07-08T12:39:33.823275+0000","last_peered":"2023-07-08T12:39:33.823275+0000","last_clean":"2023-07-08T12:39:33.823275+0000","last_became_active":"2023-07-08T12:39:31.835398+0000","last_became_peered":"2023-07-08T12:39:31.835398+0000","last_unstale":"2023-07-08T12:39:33.823275+0000","last_undegraded":"2023-07-08T12:39:33.823275+0000","last_fullsized":"2023-07-08T12:39:33.823275+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:23:32.201249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824099+0000","last_change":"2023-07-08T12:39:30.807814+0000","last_active":"2023-07-08T12:39:33.824099+0000","last_peered":"2023-07-08T12:39:33.824099+0000","last_clean":"2023-07-08T12:39:33.824099+0000","last_became_active":"2023-07-08T12:39:30.805685+0000","last_became_peered":"2023-07-08T12:39:30.805685+0000","last_unstale":"2023-07-08T12:39:33.824099+0000","last_undegraded":"2023-07-08T12:39:33.824099+0000","last_fullsized":"2023-07-08T12:39:33.824099+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:40:15.733187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832571+0000","last_change":"2023-07-08T12:39:31.839318+0000","last_active":"2023-07-08T12:39:32.832571+0000","last_peered":"2023-07-08T12:39:32.832571+0000","last_clean":"2023-07-08T12:39:32.832571+0000","last_became_active":"2023-07-08T12:39:31.838982+0000","last_became_peered":"2023-07-08T12:39:31.838982+0000","last_unstale":"2023-07-08T12:39:32.832571+0000","last_undegraded":"2023-07-08T12:39:32.832571+0000","last_fullsized":"2023-07-08T12:39:32.832571+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:42:41.668311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832574+0000","last_change":"2023-07-08T12:39:30.812696+0000","last_active":"2023-07-08T12:39:32.832574+0000","last_peered":"2023-07-08T12:39:32.832574+0000","last_clean":"2023-07-08T12:39:32.832574+0000","last_became_active":"2023-07-08T12:39:30.812416+0000","last_became_peered":"2023-07-08T12:39:30.812416+0000","last_unstale":"2023-07-08T12:39:32.832574+0000","last_undegraded":"2023-07-08T12:39:32.832574+0000","last_fullsized":"2023-07-08T12:39:32.832574+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:33:31.686095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832374+0000","last_change":"2023-07-08T12:39:31.846773+0000","last_active":"2023-07-08T12:39:32.832374+0000","last_peered":"2023-07-08T12:39:32.832374+0000","last_clean":"2023-07-08T12:39:32.832374+0000","last_became_active":"2023-07-08T12:39:31.846487+0000","last_became_peered":"2023-07-08T12:39:31.846487+0000","last_unstale":"2023-07-08T12:39:32.832374+0000","last_undegraded":"2023-07-08T12:39:32.832374+0000","last_fullsized":"2023-07-08T12:39:32.832374+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:41:40.524279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.822745+0000","last_change":"2023-07-08T12:39:30.808182+0000","last_active":"2023-07-08T12:39:33.822745+0000","last_peered":"2023-07-08T12:39:33.822745+0000","last_clean":"2023-07-08T12:39:33.822745+0000","last_became_active":"2023-07-08T12:39:30.807890+0000","last_became_peered":"2023-07-08T12:39:30.807890+0000","last_unstale":"2023-07-08T12:39:33.822745+0000","last_undegraded":"2023-07-08T12:39:33.822745+0000","last_fullsized":"2023-07-08T12:39:33.822745+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:51:59.050886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.822783+0000","last_change":"2023-07-08T12:39:31.848577+0000","last_active":"2023-07-08T12:39:33.822783+0000","last_peered":"2023-07-08T12:39:33.822783+0000","last_clean":"2023-07-08T12:39:33.822783+0000","last_became_active":"2023-07-08T12:39:31.848276+0000","last_became_peered":"2023-07-08T12:39:31.848276+0000","last_unstale":"2023-07-08T12:39:33.822783+0000","last_undegraded":"2023-07-08T12:39:33.822783+0000","last_fullsized":"2023-07-08T12:39:33.822783+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:00:36.076933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823248+0000","last_change":"2023-07-08T12:39:30.809292+0000","last_active":"2023-07-08T12:39:33.823248+0000","last_peered":"2023-07-08T12:39:33.823248+0000","last_clean":"2023-07-08T12:39:33.823248+0000","last_became_active":"2023-07-08T12:39:30.809016+0000","last_became_peered":"2023-07-08T12:39:30.809016+0000","last_unstale":"2023-07-08T12:39:33.823248+0000","last_undegraded":"2023-07-08T12:39:33.823248+0000","last_fullsized":"2023-07-08T12:39:33.823248+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:44:52.400867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823803+0000","last_change":"2023-07-08T12:39:31.844194+0000","last_active":"2023-07-08T12:39:33.823803+0000","last_peered":"2023-07-08T12:39:33.823803+0000","last_clean":"2023-07-08T12:39:33.823803+0000","last_became_active":"2023-07-08T12:39:31.843818+0000","last_became_peered":"2023-07-08T12:39:31.843818+0000","last_unstale":"2023-07-08T12:39:33.823803+0000","last_undegraded":"2023-07-08T12:39:33.823803+0000","last_fullsized":"2023-07-08T12:39:33.823803+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-10T00:13:24.762080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823157+0000","last_change":"2023-07-08T12:39:30.801673+0000","last_active":"2023-07-08T12:39:33.823157+0000","last_peered":"2023-07-08T12:39:33.823157+0000","last_clean":"2023-07-08T12:39:33.823157+0000","last_became_active":"2023-07-08T12:39:30.801308+0000","last_became_peered":"2023-07-08T12:39:30.801308+0000","last_unstale":"2023-07-08T12:39:33.823157+0000","last_undegraded":"2023-07-08T12:39:33.823157+0000","last_fullsized":"2023-07-08T12:39:33.823157+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:21:05.863482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832416+0000","last_change":"2023-07-08T12:38:03.160177+0000","last_active":"2023-07-08T12:39:32.832416+0000","last_peered":"2023-07-08T12:39:32.832416+0000","last_clean":"2023-07-08T12:39:32.832416+0000","last_became_active":"2023-07-08T12:38:03.159864+0000","last_became_peered":"2023-07-08T12:38:03.159864+0000","last_unstale":"2023-07-08T12:39:32.832416+0000","last_undegraded":"2023-07-08T12:39:32.832416+0000","last_fullsized":"2023-07-08T12:39:32.832416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:27:30.244724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824334+0000","last_change":"2023-07-08T12:39:31.844542+0000","last_active":"2023-07-08T12:39:33.824334+0000","last_peered":"2023-07-08T12:39:33.824334+0000","last_clean":"2023-07-08T12:39:33.824334+0000","last_became_active":"2023-07-08T12:39:31.844244+0000","last_became_peered":"2023-07-08T12:39:31.844244+0000","last_unstale":"2023-07-08T12:39:33.824334+0000","last_undegraded":"2023-07-08T12:39:33.824334+0000","last_fullsized":"2023-07-08T12:39:33.824334+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:39:45.583712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"42'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.829818+0000","last_change":"2023-07-08T12:39:30.798685+0000","last_active":"2023-07-08T12:39:33.829818+0000","last_peered":"2023-07-08T12:39:33.829818+0000","last_clean":"2023-07-08T12:39:33.829818+0000","last_became_active":"2023-07-08T12:39:30.798404+0000","last_became_peered":"2023-07-08T12:39:30.798404+0000","last_unstale":"2023-07-08T12:39:33.829818+0000","last_undegraded":"2023-07-08T12:39:33.829818+0000","last_fullsized":"2023-07-08T12:39:33.829818+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:43:36.645028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":55,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832671+0000","last_change":"2023-07-08T12:38:02.384639+0000","last_active":"2023-07-08T12:39:32.832671+0000","last_peered":"2023-07-08T12:39:32.832671+0000","last_clean":"2023-07-08T12:39:32.832671+0000","last_became_active":"2023-07-08T12:38:02.384372+0000","last_became_peered":"2023-07-08T12:38:02.384372+0000","last_unstale":"2023-07-08T12:39:32.832671+0000","last_undegraded":"2023-07-08T12:39:32.832671+0000","last_fullsized":"2023-07-08T12:39:32.832671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:38:47.309288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823439+0000","last_change":"2023-07-08T12:39:31.850608+0000","last_active":"2023-07-08T12:39:33.823439+0000","last_peered":"2023-07-08T12:39:33.823439+0000","last_clean":"2023-07-08T12:39:33.823439+0000","last_became_active":"2023-07-08T12:39:31.850318+0000","last_became_peered":"2023-07-08T12:39:31.850318+0000","last_unstale":"2023-07-08T12:39:33.823439+0000","last_undegraded":"2023-07-08T12:39:33.823439+0000","last_fullsized":"2023-07-08T12:39:33.823439+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T15:33:32.544109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823789+0000","last_change":"2023-07-08T12:39:30.812605+0000","last_active":"2023-07-08T12:39:33.823789+0000","last_peered":"2023-07-08T12:39:33.823789+0000","last_clean":"2023-07-08T12:39:33.823789+0000","last_became_active":"2023-07-08T12:39:30.812321+0000","last_became_peered":"2023-07-08T12:39:30.812321+0000","last_unstale":"2023-07-08T12:39:33.823789+0000","last_undegraded":"2023-07-08T12:39:33.823789+0000","last_fullsized":"2023-07-08T12:39:33.823789+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T18:26:10.669381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.831807+0000","last_change":"2023-07-08T12:38:03.212466+0000","last_active":"2023-07-08T12:39:32.831807+0000","last_peered":"2023-07-08T12:39:32.831807+0000","last_clean":"2023-07-08T12:39:32.831807+0000","last_became_active":"2023-07-08T12:38:03.212182+0000","last_became_peered":"2023-07-08T12:38:03.212182+0000","last_unstale":"2023-07-08T12:39:32.831807+0000","last_undegraded":"2023-07-08T12:39:32.831807+0000","last_fullsized":"2023-07-08T12:39:32.831807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:36:05.121988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823909+0000","last_change":"2023-07-08T12:39:31.847941+0000","last_active":"2023-07-08T12:39:33.823909+0000","last_peered":"2023-07-08T12:39:33.823909+0000","last_clean":"2023-07-08T12:39:33.823909+0000","last_became_active":"2023-07-08T12:39:31.847161+0000","last_became_peered":"2023-07-08T12:39:31.847161+0000","last_unstale":"2023-07-08T12:39:33.823909+0000","last_undegraded":"2023-07-08T12:39:33.823909+0000","last_fullsized":"2023-07-08T12:39:33.823909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:34:46.998464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.833417+0000","last_change":"2023-07-08T12:39:30.802899+0000","last_active":"2023-07-08T12:39:32.833417+0000","last_peered":"2023-07-08T12:39:32.833417+0000","last_clean":"2023-07-08T12:39:32.833417+0000","last_became_active":"2023-07-08T12:39:30.802622+0000","last_became_peered":"2023-07-08T12:39:30.802622+0000","last_unstale":"2023-07-08T12:39:32.833417+0000","last_undegraded":"2023-07-08T12:39:32.833417+0000","last_fullsized":"2023-07-08T12:39:32.833417+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:59:30.340922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":55,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832844+0000","last_change":"2023-07-08T12:38:02.380175+0000","last_active":"2023-07-08T12:39:32.832844+0000","last_peered":"2023-07-08T12:39:32.832844+0000","last_clean":"2023-07-08T12:39:32.832844+0000","last_became_active":"2023-07-08T12:38:02.379867+0000","last_became_peered":"2023-07-08T12:38:02.379867+0000","last_unstale":"2023-07-08T12:39:32.832844+0000","last_undegraded":"2023-07-08T12:39:32.832844+0000","last_fullsized":"2023-07-08T12:39:32.832844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:43:23.976967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":209,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823430+0000","last_change":"2023-07-08T12:37:58.222371+0000","last_active":"2023-07-08T12:39:33.823430+0000","last_peered":"2023-07-08T12:39:33.823430+0000","last_clean":"2023-07-08T12:39:33.823430+0000","last_became_active":"2023-07-08T12:37:58.221381+0000","last_became_peered":"2023-07-08T12:37:58.221381+0000","last_unstale":"2023-07-08T12:39:33.823430+0000","last_undegraded":"2023-07-08T12:39:33.823430+0000","last_fullsized":"2023-07-08T12:39:33.823430+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:37:56.327863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:37:56.327863+0000","last_clean_scrub_stamp":"2023-07-08T12:37:56.327863+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T16:55:48.218985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823549+0000","last_change":"2023-07-08T12:39:31.833655+0000","last_active":"2023-07-08T12:39:33.823549+0000","last_peered":"2023-07-08T12:39:33.823549+0000","last_clean":"2023-07-08T12:39:33.823549+0000","last_became_active":"2023-07-08T12:39:31.832834+0000","last_became_peered":"2023-07-08T12:39:31.832834+0000","last_unstale":"2023-07-08T12:39:33.823549+0000","last_undegraded":"2023-07-08T12:39:33.823549+0000","last_fullsized":"2023-07-08T12:39:33.823549+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:59:21.159179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832287+0000","last_change":"2023-07-08T12:39:30.805510+0000","last_active":"2023-07-08T12:39:32.832287+0000","last_peered":"2023-07-08T12:39:32.832287+0000","last_clean":"2023-07-08T12:39:32.832287+0000","last_became_active":"2023-07-08T12:39:30.805181+0000","last_became_peered":"2023-07-08T12:39:30.805181+0000","last_unstale":"2023-07-08T12:39:32.832287+0000","last_undegraded":"2023-07-08T12:39:32.832287+0000","last_fullsized":"2023-07-08T12:39:32.832287+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T21:23:22.320168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823579+0000","last_change":"2023-07-08T12:38:02.385193+0000","last_active":"2023-07-08T12:39:33.823579+0000","last_peered":"2023-07-08T12:39:33.823579+0000","last_clean":"2023-07-08T12:39:33.823579+0000","last_became_active":"2023-07-08T12:38:02.384865+0000","last_became_peered":"2023-07-08T12:38:02.384865+0000","last_unstale":"2023-07-08T12:39:33.823579+0000","last_undegraded":"2023-07-08T12:39:33.823579+0000","last_fullsized":"2023-07-08T12:39:33.823579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:58:01.917022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832718+0000","last_change":"2023-07-08T12:39:31.837700+0000","last_active":"2023-07-08T12:39:32.832718+0000","last_peered":"2023-07-08T12:39:32.832718+0000","last_clean":"2023-07-08T12:39:32.832718+0000","last_became_active":"2023-07-08T12:39:31.837414+0000","last_became_peered":"2023-07-08T12:39:31.837414+0000","last_unstale":"2023-07-08T12:39:32.832718+0000","last_undegraded":"2023-07-08T12:39:32.832718+0000","last_fullsized":"2023-07-08T12:39:32.832718+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T19:56:30.242464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832734+0000","last_change":"2023-07-08T12:39:30.807560+0000","last_active":"2023-07-08T12:39:32.832734+0000","last_peered":"2023-07-08T12:39:32.832734+0000","last_clean":"2023-07-08T12:39:32.832734+0000","last_became_active":"2023-07-08T12:39:30.807213+0000","last_became_peered":"2023-07-08T12:39:30.807213+0000","last_unstale":"2023-07-08T12:39:32.832734+0000","last_undegraded":"2023-07-08T12:39:32.832734+0000","last_fullsized":"2023-07-08T12:39:32.832734+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T17:25:37.748428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823916+0000","last_change":"2023-07-08T12:38:03.160344+0000","last_active":"2023-07-08T12:39:33.823916+0000","last_peered":"2023-07-08T12:39:33.823916+0000","last_clean":"2023-07-08T12:39:33.823916+0000","last_became_active":"2023-07-08T12:38:03.160009+0000","last_became_peered":"2023-07-08T12:38:03.160009+0000","last_unstale":"2023-07-08T12:39:33.823916+0000","last_undegraded":"2023-07-08T12:39:33.823916+0000","last_fullsized":"2023-07-08T12:39:33.823916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T12:39:38.000178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824041+0000","last_change":"2023-07-08T12:39:31.851725+0000","last_active":"2023-07-08T12:39:33.824041+0000","last_peered":"2023-07-08T12:39:33.824041+0000","last_clean":"2023-07-08T12:39:33.824041+0000","last_became_active":"2023-07-08T12:39:31.846550+0000","last_became_peered":"2023-07-08T12:39:31.846550+0000","last_unstale":"2023-07-08T12:39:33.824041+0000","last_undegraded":"2023-07-08T12:39:33.824041+0000","last_fullsized":"2023-07-08T12:39:33.824041+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:45:18.051273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824390+0000","last_change":"2023-07-08T12:39:30.799831+0000","last_active":"2023-07-08T12:39:33.824390+0000","last_peered":"2023-07-08T12:39:33.824390+0000","last_clean":"2023-07-08T12:39:33.824390+0000","last_became_active":"2023-07-08T12:39:30.799467+0000","last_became_peered":"2023-07-08T12:39:30.799467+0000","last_unstale":"2023-07-08T12:39:33.824390+0000","last_undegraded":"2023-07-08T12:39:33.824390+0000","last_fullsized":"2023-07-08T12:39:33.824390+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T20:21:40.461227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:32.832851+0000","last_change":"2023-07-08T12:38:02.385821+0000","last_active":"2023-07-08T12:39:32.832851+0000","last_peered":"2023-07-08T12:39:32.832851+0000","last_clean":"2023-07-08T12:39:32.832851+0000","last_became_active":"2023-07-08T12:38:02.385512+0000","last_became_peered":"2023-07-08T12:38:02.385512+0000","last_unstale":"2023-07-08T12:39:32.832851+0000","last_undegraded":"2023-07-08T12:39:32.832851+0000","last_fullsized":"2023-07-08T12:39:32.832851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T13:09:20.893204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823790+0000","last_change":"2023-07-08T12:39:31.842293+0000","last_active":"2023-07-08T12:39:33.823790+0000","last_peered":"2023-07-08T12:39:33.823790+0000","last_clean":"2023-07-08T12:39:33.823790+0000","last_became_active":"2023-07-08T12:39:31.841991+0000","last_became_peered":"2023-07-08T12:39:31.841991+0000","last_unstale":"2023-07-08T12:39:33.823790+0000","last_undegraded":"2023-07-08T12:39:33.823790+0000","last_fullsized":"2023-07-08T12:39:33.823790+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:30.782334+0000","last_clean_scrub_stamp":"2023-07-08T12:39:30.782334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T22:47:24.720020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.823503+0000","last_change":"2023-07-08T12:39:30.812718+0000","last_active":"2023-07-08T12:39:33.823503+0000","last_peered":"2023-07-08T12:39:33.823503+0000","last_clean":"2023-07-08T12:39:33.823503+0000","last_became_active":"2023-07-08T12:39:30.812332+0000","last_became_peered":"2023-07-08T12:39:30.812332+0000","last_unstale":"2023-07-08T12:39:33.823503+0000","last_undegraded":"2023-07-08T12:39:33.823503+0000","last_fullsized":"2023-07-08T12:39:33.823503+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:39:29.780305+0000","last_clean_scrub_stamp":"2023-07-08T12:39:29.780305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T23:17:46.024443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-07-08T12:39:33.824669+0000","last_change":"2023-07-08T12:38:02.385388+0000","last_active":"2023-07-08T12:39:33.824669+0000","last_peered":"2023-07-08T12:39:33.824669+0000","last_clean":"2023-07-08T12:39:33.824669+0000","last_became_active":"2023-07-08T12:38:02.385110+0000","last_became_peered":"2023-07-08T12:38:02.385110+0000","last_unstale":"2023-07-08T12:39:33.824669+0000","last_undegraded":"2023-07-08T12:39:33.824669+0000","last_fullsized":"2023-07-08T12:39:33.824669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-08T12:38:01.356406+0000","last_clean_scrub_stamp":"2023-07-08T12:38:01.356406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-09T14:53:46.409836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1085440,"data_stored":807909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.69499999999999995}]},{"osd":1,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.66200000000000003}]},{"osd":2,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.625},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.58199999999999996}]},{"osd":3,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.68200000000000005}]},{"osd":4,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.379},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.45000000000000001}]},{"osd":5,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.47399999999999998}]},{"osd":6,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.434},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.51500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":495616,"data_stored":218123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.61099999999999999}]},{"osd":1,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.42699999999999999}]},{"osd":2,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":11.653,"5min":11.653,"15min":11.653},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.38100000000000001}]},{"osd":3,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.47199999999999998}]},{"osd":4,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.5},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.44700000000000001}]},{"osd":7,"last update":"Sat Jul 8 12:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":491520,"data_stored":217521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.315}]},{"osd":2,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.38}]},{"osd":3,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.42399999999999999}]},{"osd":4,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.70099999999999996}]},{"osd":5,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67800000000000005}]},{"osd":6,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.77100000000000002}]},{"osd":7,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.71999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1085440,"data_stored":807853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":10.974,"5min":10.974,"15min":10.974},"last":0.55000000000000004}]},{"osd":2,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.433}]},{"osd":3,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.46000000000000002}]},{"osd":4,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.992000000000001,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.60799999999999998}]},{"osd":5,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.73099999999999998}]},{"osd":6,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":12.029,"5min":12.029,"15min":12.029},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Sat Jul 8 12:38:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":11.941000000000001,"5min":11.941000000000001,"15min":11.941000000000001},"last":0.64200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":503808,"data_stored":219037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.37}]},{"osd":1,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.45400000000000001}]},{"osd":4,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.70099999999999996}]},{"osd":5,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.878},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.83099999999999996}]},{"osd":6,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.81499999999999995}]},{"osd":7,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.85399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":487424,"data_stored":211116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.371}]},{"osd":1,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.39600000000000002}]},{"osd":2,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.45100000000000001}]},{"osd":4,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.69699999999999995}]},{"osd":5,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.66200000000000003}]},{"osd":6,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71899999999999997}]},{"osd":7,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.746},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.75700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":499712,"data_stored":219015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.47999999999999998}]},{"osd":1,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.33800000000000002}]},{"osd":2,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.53100000000000003}]},{"osd":3,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.497},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.627}]},{"osd":5,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.206,"5min":3.206,"15min":3.206},"last":0.60299999999999998}]},{"osd":6,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.57299999999999995}]},{"osd":7,"last update":"Sat Jul 8 12:38:56 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.859,"5min":3.859,"15min":3.859},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.65400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":491520,"data_stored":217521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.42299999999999999}]},{"osd":1,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.51700000000000002}]},{"osd":2,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.501},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68500000000000005}]},{"osd":3,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.71199999999999997}]},{"osd":4,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.63}]},{"osd":6,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.73199999999999998}]},{"osd":7,"last update":"Sat Jul 8 12:38:57 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-08T12:39:36.369 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-08T12:39:36.370 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-08T12:39:36.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.532+0000 7f68ab00e700 1 Processor -- start 2023-07-08T12:39:36.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.532+0000 7f68ab00e700 1 -- start start 2023-07-08T12:39:36.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68ab00e700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f68a40aeec0 0x7f68a406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68ab00e700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f68a40cdb80 0x7f68a40cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68ab00e700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a415d770 con 0x7f68a406ebe0 2023-07-08T12:39:36.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68ab00e700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a415a600 con 0x7f68a40cdb80 2023-07-08T12:39:36.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68ab00e700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a415cb60 con 0x7f68a40aeec0 2023-07-08T12:39:36.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68a95ab700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f68a40cdb80 0x7f68a40cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68a3fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f68a406ebe0 0x7f68a40cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39238/0 (socket says 172.21.15.12:39238) 2023-07-08T12:39:36.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68a3fff700 1 -- 172.21.15.12:0/824075137 learned_addr learned my addr 172.21.15.12:0/824075137 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:36.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68a8daa700 1 --2- 172.21.15.12:0/824075137 >> v2:172.21.15.160:3300/0 conn(0x7f68a40aeec0 0x7f68a406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68a95ab700 1 -- 172.21.15.12:0/824075137 >> v2:172.21.15.160:3300/0 conn(0x7f68a40aeec0 msgr2=0x7f68a406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:36.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68a95ab700 1 --2- 172.21.15.12:0/824075137 >> v2:172.21.15.160:3300/0 conn(0x7f68a40aeec0 0x7f68a406cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68a95ab700 1 -- 172.21.15.12:0/824075137 >> v1:172.21.15.12:6789/0 conn(0x7f68a406ebe0 legacy=0x7f68a40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68a37fe700 1 -- 172.21.15.12:0/824075137 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154305493 0 0) 0x7f68a415d770 con 0x7f68a406ebe0 2023-07-08T12:39:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.533+0000 7f68a95ab700 1 -- 172.21.15.12:0/824075137 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a415ce40 con 0x7f68a40cdb80 2023-07-08T12:39:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.534+0000 7f68a95ab700 1 --2- 172.21.15.12:0/824075137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a40cdb80 0x7f68a40cbe10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f68a40b23f0 tx=0x7f689c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76474c84acc5ef04 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.534+0000 7f68a37fe700 1 -- 172.21.15.12:0/824075137 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c00e070 con 0x7f68a40cdb80 2023-07-08T12:39:36.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.534+0000 7f68a37fe700 1 -- 172.21.15.12:0/824075137 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c00bbd0 con 0x7f68a40cdb80 2023-07-08T12:39:36.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.534+0000 7f68a37fe700 1 -- 172.21.15.12:0/824075137 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c012430 con 0x7f68a40cdb80 2023-07-08T12:39:36.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.534+0000 7f68ab00e700 1 -- 172.21.15.12:0/824075137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a40cdb80 msgr2=0x7f68a40cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.534+0000 7f68ab00e700 1 --2- 172.21.15.12:0/824075137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a40cdb80 0x7f68a40cbe10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f68a40b23f0 tx=0x7f689c00b330 comp rx=0 tx=0).stop 2023-07-08T12:39:36.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.534+0000 7f68ab00e700 1 -- 172.21.15.12:0/824075137 shutdown_connections 2023-07-08T12:39:36.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.534+0000 7f68ab00e700 1 --2- 172.21.15.12:0/824075137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a40cdb80 0x7f68a40cbe10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.534+0000 7f68ab00e700 1 --2- 172.21.15.12:0/824075137 >> v2:172.21.15.160:3300/0 conn(0x7f68a40aeec0 0x7f68a406cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.534+0000 7f68ab00e700 1 -- 172.21.15.12:0/824075137 >> 172.21.15.12:0/824075137 conn(0x7f68a4098b50 msgr2=0x7f68a406b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:36.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.534+0000 7f68ab00e700 1 -- 172.21.15.12:0/824075137 shutdown_connections 2023-07-08T12:39:36.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.534+0000 7f68ab00e700 1 -- 172.21.15.12:0/824075137 wait complete. 2023-07-08T12:39:36.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.536+0000 7f68ab00e700 1 Processor -- start 2023-07-08T12:39:36.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.536+0000 7f68ab00e700 1 -- start start 2023-07-08T12:39:36.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.536+0000 7f68ab00e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68a406ebe0 0x7f68a41f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.536+0000 7f68ab00e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68a40aeec0 0x7f68a4157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68ab00e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a40cdb80 0x7f68a4157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68ab00e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a40c4af0 con 0x7f68a40cdb80 2023-07-08T12:39:36.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68ab00e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a4151710 con 0x7f68a406ebe0 2023-07-08T12:39:36.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68ab00e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a40c3e70 con 0x7f68a40aeec0 2023-07-08T12:39:36.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a95ab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a40cdb80 0x7f68a4157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a95ab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a40cdb80 0x7f68a4157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51852/0 (socket says 172.21.15.12:51852) 2023-07-08T12:39:36.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a95ab700 1 -- 172.21.15.12:0/1299986213 learned_addr learned my addr 172.21.15.12:0/1299986213 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:36.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a8daa700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68a406ebe0 0x7f68a41f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a3fff700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68a40aeec0 0x7f68a4157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a95ab700 1 -- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68a40aeec0 msgr2=0x7f68a4157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a95ab700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68a40aeec0 0x7f68a4157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a95ab700 1 -- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68a406ebe0 msgr2=0x7f68a41f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a95ab700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68a406ebe0 0x7f68a41f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a95ab700 1 -- 172.21.15.12:0/1299986213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c009a70 con 0x7f68a40cdb80 2023-07-08T12:39:36.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a95ab700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a40cdb80 0x7f68a4157b70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f689c013000 tx=0x7f689c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:36.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a17fa700 1 -- 172.21.15.12:0/1299986213 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c012430 con 0x7f68a40cdb80 2023-07-08T12:39:36.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a17fa700 1 -- 172.21.15.12:0/1299986213 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c020070 con 0x7f68a40cdb80 2023-07-08T12:39:36.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68a17fa700 1 -- 172.21.15.12:0/1299986213 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c004280 con 0x7f68a40cdb80 2023-07-08T12:39:36.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68ab00e700 1 -- 172.21.15.12:0/1299986213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a40c4150 con 0x7f68a40cdb80 2023-07-08T12:39:36.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.537+0000 7f68ab00e700 1 -- 172.21.15.12:0/1299986213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a40c4710 con 0x7f68a40cdb80 2023-07-08T12:39:36.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.538+0000 7f68ab00e700 1 -- 172.21.15.12:0/1299986213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a40c4c30 con 0x7f68a40cdb80 2023-07-08T12:39:36.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.539+0000 7f68a17fa700 1 -- 172.21.15.12:0/1299986213 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f689c0043e0 con 0x7f68a40cdb80 2023-07-08T12:39:36.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.539+0000 7f68a17fa700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f688806ace0 0x7f688806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.539+0000 7f68a17fa700 1 -- 172.21.15.12:0/1299986213 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f689c022070 con 0x7f68a40cdb80 2023-07-08T12:39:36.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.539+0000 7f68a8daa700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f688806ace0 0x7f688806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.540+0000 7f68a8daa700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f688806ace0 0x7f688806d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6898006fd0 tx=0x7f6898008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:36.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.544+0000 7f68a17fa700 1 -- 172.21.15.12:0/1299986213 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f68a40c4c30 con 0x7f68a40cdb80 2023-07-08T12:39:36.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.825+0000 7f68ab00e700 1 -- 172.21.15.12:0/1299986213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f68a40c4c30 con 0x7f68a40cdb80 2023-07-08T12:39:36.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.829+0000 7f68a17fa700 1 -- 172.21.15.12:0/1299986213 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f68a40c4c30 con 0x7f68a40cdb80 2023-07-08T12:39:36.831 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2023-07-08T12:39:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB0WKlkvys7OhAA7XADn3L9fMd0Gow4lK3U0g== 2023-07-08T12:39:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: caps mds = "allow" 2023-07-08T12:39:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: caps mon = "allow r" 2023-07-08T12:39:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-08T12:39:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.831+0000 7f68ab00e700 1 -- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f688806ace0 msgr2=0x7f688806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.831+0000 7f68ab00e700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f688806ace0 0x7f688806d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6898006fd0 tx=0x7f6898008040 comp rx=0 tx=0).stop 2023-07-08T12:39:36.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.831+0000 7f68ab00e700 1 -- 172.21.15.12:0/1299986213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a40cdb80 msgr2=0x7f68a4157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.831+0000 7f68ab00e700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a40cdb80 0x7f68a4157b70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f689c013000 tx=0x7f689c004ad0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.831+0000 7f68ab00e700 1 -- 172.21.15.12:0/1299986213 shutdown_connections 2023-07-08T12:39:36.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.831+0000 7f68ab00e700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f688806ace0 0x7f688806d190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.831+0000 7f68ab00e700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a40cdb80 0x7f68a4157b70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.831+0000 7f68ab00e700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68a406ebe0 0x7f68a41f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.831+0000 7f68ab00e700 1 --2- 172.21.15.12:0/1299986213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68a40aeec0 0x7f68a4157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.831+0000 7f68ab00e700 1 -- 172.21.15.12:0/1299986213 >> 172.21.15.12:0/1299986213 conn(0x7f68a4098b50 msgr2=0x7f68a415bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.831+0000 7f68ab00e700 1 -- 172.21.15.12:0/1299986213 shutdown_connections 2023-07-08T12:39:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.831+0000 7f68ab00e700 1 -- 172.21.15.12:0/1299986213 wait complete. 2023-07-08T12:39:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:updated caps for client.0 2023-07-08T12:39:36.846 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-08T12:39:36.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.968+0000 7fb3e5d22700 1 Processor -- start 2023-07-08T12:39:36.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.968+0000 7fb3e5d22700 1 -- start start 2023-07-08T12:39:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.969+0000 7fb3e5d22700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb3e0159c30 0x7fb3e00cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.969+0000 7fb3e5d22700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e015c890 con 0x7fb3e0160790 2023-07-08T12:39:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.969+0000 7fb3e5d22700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e006f620 con 0x7fb3e0069f30 2023-07-08T12:39:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.969+0000 7fb3e5d22700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e015c730 con 0x7fb3e0159c30 2023-07-08T12:39:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.969+0000 7fb3deffd700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb3e0160790 0x7fb3e006ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39254/0 (socket says 172.21.15.12:39254) 2023-07-08T12:39:36.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.969+0000 7fb3deffd700 1 -- 172.21.15.12:0/1991253460 learned_addr learned my addr 172.21.15.12:0/1991253460 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3df7fe700 1 --2- 172.21.15.12:0/1991253460 >> v2:172.21.15.160:3300/0 conn(0x7fb3e0159c30 0x7fb3e00cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1229601847 0 0) 0x7fb3e015c890 con 0x7fb3e0160790 2023-07-08T12:39:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3cc0035b0 con 0x7fb3e0160790 2023-07-08T12:39:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480176776 0 0) 0x7fb3e006f620 con 0x7fb3e0069f30 2023-07-08T12:39:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3e015c890 con 0x7fb3e0069f30 2023-07-08T12:39:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3172734713 0 0) 0x7fb3cc0035b0 con 0x7fb3e0160790 2023-07-08T12:39:36.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 >> v2:172.21.15.160:3300/0 conn(0x7fb3e0159c30 msgr2=0x7fb3e00cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 --2- 172.21.15.12:0/1991253460 >> v2:172.21.15.160:3300/0 conn(0x7fb3e0159c30 0x7fb3e00cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 >> v1:172.21.15.160:6789/0 conn(0x7fb3e0069f30 legacy=0x7fb3e015cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e0160c90 con 0x7fb3e0160790 2023-07-08T12:39:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7fb3c8001670 con 0x7fb3e0160790 2023-07-08T12:39:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 >> v1:172.21.15.12:6789/0 conn(0x7fb3e0160790 legacy=0x7fb3e006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 --2- 172.21.15.12:0/1991253460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3cc003b90 0x7fb3cc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 --2- 172.21.15.12:0/1991253460 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3cc004850 0x7fb3cc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 --2- 172.21.15.12:0/1991253460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3cc005200 0x7fb3cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e015c730 con 0x7fb3cc005200 2023-07-08T12:39:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc00c060 con 0x7fb3cc003b90 2023-07-08T12:39:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc00c220 con 0x7fb3cc004850 2023-07-08T12:39:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3dffff700 1 --2- 172.21.15.12:0/1991253460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3cc005200 0x7fb3cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3df7fe700 1 --2- 172.21.15.12:0/1991253460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3cc003b90 0x7fb3cc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3deffd700 1 --2- 172.21.15.12:0/1991253460 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3cc004850 0x7fb3cc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3dffff700 1 -- 172.21.15.12:0/1991253460 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3cc004850 msgr2=0x7fb3cc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.970+0000 7fb3dffff700 1 --2- 172.21.15.12:0/1991253460 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3cc004850 0x7fb3cc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3dffff700 1 -- 172.21.15.12:0/1991253460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3cc003b90 msgr2=0x7fb3cc003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3dffff700 1 --2- 172.21.15.12:0/1991253460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3cc003b90 0x7fb3cc003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3dffff700 1 -- 172.21.15.12:0/1991253460 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc0035b0 con 0x7fb3cc005200 2023-07-08T12:39:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3dffff700 1 -- 172.21.15.12:0/1991253460 reap_dead start 2023-07-08T12:39:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3dffff700 1 --2- 172.21.15.12:0/1991253460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3cc005200 0x7fb3cc00ba10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800c2f0 tx=0x7fb3cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef780457a2ffc30 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8001f30 con 0x7fb3cc005200 2023-07-08T12:39:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d8003a00 con 0x7fb3cc005200 2023-07-08T12:39:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3de7fc700 1 -- 172.21.15.12:0/1991253460 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8003db0 con 0x7fb3cc005200 2023-07-08T12:39:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3e5d22700 1 -- 172.21.15.12:0/1991253460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3cc005200 msgr2=0x7fb3cc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3e5d22700 1 --2- 172.21.15.12:0/1991253460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3cc005200 0x7fb3cc00ba10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800c2f0 tx=0x7fb3cc005880 comp rx=0 tx=0).stop 2023-07-08T12:39:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3e5d22700 1 -- 172.21.15.12:0/1991253460 shutdown_connections 2023-07-08T12:39:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3e5d22700 1 --2- 172.21.15.12:0/1991253460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3cc005200 0x7fb3cc00ba10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3e5d22700 1 -- 172.21.15.12:0/1991253460 >> 172.21.15.12:0/1991253460 conn(0x7fb3e00cc290 msgr2=0x7fb3e00cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3e5d22700 1 -- 172.21.15.12:0/1991253460 shutdown_connections 2023-07-08T12:39:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.971+0000 7fb3e5d22700 1 -- 172.21.15.12:0/1991253460 wait complete. 2023-07-08T12:39:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.973+0000 7fb3e5d22700 1 Processor -- start 2023-07-08T12:39:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.973+0000 7fb3e5d22700 1 -- start start 2023-07-08T12:39:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.973+0000 7fb3e5d22700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3cc005200 0x7fb3e00c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3e5d22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e00ac230 0x7fb3e01510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3e5d22700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3e015e150 0x7fb3e015e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3e5d22700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e015eb00 con 0x7fb3e00ac230 2023-07-08T12:39:36.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3e5d22700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00c4fa0 con 0x7fb3e015e150 2023-07-08T12:39:36.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3e5d22700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00c5de0 con 0x7fb3cc005200 2023-07-08T12:39:36.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3deffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e00ac230 0x7fb3e01510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3deffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e00ac230 0x7fb3e01510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51870/0 (socket says 172.21.15.12:51870) 2023-07-08T12:39:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3deffd700 1 -- 172.21.15.12:0/3597078026 learned_addr learned my addr 172.21.15.12:0/3597078026 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3df7fe700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3cc005200 0x7fb3e00c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3dffff700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3e015e150 0x7fb3e015e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3deffd700 1 -- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3cc005200 msgr2=0x7fb3e00c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3deffd700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3cc005200 0x7fb3e00c4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3deffd700 1 -- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3e015e150 msgr2=0x7fb3e015e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3deffd700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3e015e150 0x7fb3e015e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3deffd700 1 -- 172.21.15.12:0/3597078026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00c6270 con 0x7fb3e00ac230 2023-07-08T12:39:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3dffff700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3e015e150 0x7fb3e015e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3deffd700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e00ac230 0x7fb3e01510e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8007310 tx=0x7fb3c8007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3d7fff700 1 -- 172.21.15.12:0/3597078026 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800f660 con 0x7fb3e00ac230 2023-07-08T12:39:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3d7fff700 1 -- 172.21.15.12:0/3597078026 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c800f820 con 0x7fb3e00ac230 2023-07-08T12:39:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3e5d22700 1 -- 172.21.15.12:0/3597078026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e00c51d0 con 0x7fb3e00ac230 2023-07-08T12:39:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3d7fff700 1 -- 172.21.15.12:0/3597078026 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800fbd0 con 0x7fb3e00ac230 2023-07-08T12:39:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.974+0000 7fb3e5d22700 1 -- 172.21.15.12:0/3597078026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e00c5790 con 0x7fb3e00ac230 2023-07-08T12:39:36.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.975+0000 7fb3e5d22700 1 -- 172.21.15.12:0/3597078026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e00c5f50 con 0x7fb3e00ac230 2023-07-08T12:39:36.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.976+0000 7fb3d7fff700 1 -- 172.21.15.12:0/3597078026 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb3c800fd30 con 0x7fb3e00ac230 2023-07-08T12:39:36.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.976+0000 7fb3d7fff700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb3c006ace0 0x7fb3c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:36.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.976+0000 7fb3d7fff700 1 -- 172.21.15.12:0/3597078026 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb3c8089500 con 0x7fb3e00ac230 2023-07-08T12:39:36.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.977+0000 7fb3df7fe700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb3c006ace0 0x7fb3c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:36.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.977+0000 7fb3df7fe700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb3c006ace0 0x7fb3c006d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009b20 tx=0x7fb3d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:36.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:36.981+0000 7fb3d7fff700 1 -- 172.21.15.12:0/3597078026 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb3c8054500 con 0x7fb3e00ac230 2023-07-08T12:39:37.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.225+0000 7fb3e5d22700 1 -- 172.21.15.12:0/3597078026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3e004ea60 con 0x7fb3e00ac230 2023-07-08T12:39:37.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.225+0000 7fb3d7fff700 1 -- 172.21.15.12:0/3597078026 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4671 (secure 0 0 0) 0x7fb3c8057b20 con 0x7fb3e00ac230 2023-07-08T12:39:37.228 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:37.228 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4937,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/1217176712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":1217176712},{"type":"v1","addr":"172.21.15.160:6836","nonce":1217176712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5215,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/396620950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":396620950},{"type":"v1","addr":"172.21.15.160:6837","nonce":396620950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5343,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6834/1725416152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1725416152},{"type":"v1","addr":"172.21.15.12:6834","nonce":1725416152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-08T12:39:31.213203+0000","modified":"2023-07-08T12:39:34.835529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5346},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5346":{"gid":5346,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.12:6835/590221748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6833","nonce":590221748},{"type":"v1","addr":"172.21.15.12:6835","nonce":590221748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-08T12:39:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.227+0000 7fb3e5d22700 1 -- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb3c006ace0 msgr2=0x7fb3c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.227+0000 7fb3e5d22700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb3c006ace0 0x7fb3c006d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009b20 tx=0x7fb3d0008040 comp rx=0 tx=0).stop 2023-07-08T12:39:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.227+0000 7fb3e5d22700 1 -- 172.21.15.12:0/3597078026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e00ac230 msgr2=0x7fb3e01510e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.227+0000 7fb3e5d22700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e00ac230 0x7fb3e01510e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8007310 tx=0x7fb3c8007620 comp rx=0 tx=0).stop 2023-07-08T12:39:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.228+0000 7fb3e5d22700 1 -- 172.21.15.12:0/3597078026 shutdown_connections 2023-07-08T12:39:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.228+0000 7fb3e5d22700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7fb3c006ace0 0x7fb3c006d190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.228+0000 7fb3e5d22700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3e015e150 0x7fb3e015e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.228+0000 7fb3e5d22700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3cc005200 0x7fb3e00c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.228+0000 7fb3e5d22700 1 --2- 172.21.15.12:0/3597078026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3e00ac230 0x7fb3e01510e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.228+0000 7fb3e5d22700 1 -- 172.21.15.12:0/3597078026 >> 172.21.15.12:0/3597078026 conn(0x7fb3e00cc290 msgr2=0x7fb3e015be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.228+0000 7fb3e5d22700 1 -- 172.21.15.12:0/3597078026 shutdown_connections 2023-07-08T12:39:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.228+0000 7fb3e5d22700 1 -- 172.21.15.12:0/3597078026 wait complete. 2023-07-08T12:39:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-07-08T12:39:37.644 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-08T12:39:31.213203+0000', 'modified': '2023-07-08T12:39:34.835529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5346}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5346': {'gid': 5346, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.12:6835/590221748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6833', 'nonce': 590221748}, {'type': 'v1', 'addr': '172.21.15.12:6835', 'nonce': 590221748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-08T12:39:37.644 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-08T12:39:37.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-08T12:39:37.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.774+0000 7f05589a0700 1 Processor -- start 2023-07-08T12:39:37.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.775+0000 7f05589a0700 1 -- start start 2023-07-08T12:39:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.775+0000 7f05589a0700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f05540b5f90 0x7f05540cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.775+0000 7f05589a0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f05540cd4d0 0x7f05540ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.775+0000 7f05589a0700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05540afa50 con 0x7f05540ce820 2023-07-08T12:39:37.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.775+0000 7f05589a0700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05540d0560 con 0x7f05540cd4d0 2023-07-08T12:39:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.775+0000 7f05589a0700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05540aff80 con 0x7f05540b5f90 2023-07-08T12:39:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.775+0000 7f0552ffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f05540cd4d0 0x7f05540ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.776+0000 7f0552ffd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f05540cd4d0 0x7f05540ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51876/0 (socket says 172.21.15.12:51876) 2023-07-08T12:39:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.776+0000 7f0552ffd700 1 -- 172.21.15.12:0/2691523348 learned_addr learned my addr 172.21.15.12:0/2691523348 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.776+0000 7f05537fe700 1 --2- 172.21.15.12:0/2691523348 >> v2:172.21.15.160:3300/0 conn(0x7f05540b5f90 0x7f05540cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.776+0000 7f0552ffd700 1 -- 172.21.15.12:0/2691523348 >> v2:172.21.15.160:3300/0 conn(0x7f05540b5f90 msgr2=0x7f05540cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:37.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.776+0000 7f0552ffd700 1 --2- 172.21.15.12:0/2691523348 >> v2:172.21.15.160:3300/0 conn(0x7f05540b5f90 0x7f05540cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.776+0000 7f0552ffd700 1 -- 172.21.15.12:0/2691523348 >> v1:172.21.15.160:6789/0 conn(0x7f05540ce820 legacy=0x7f055415dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:37.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.776+0000 7f0552ffd700 1 -- 172.21.15.12:0/2691523348 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05540cffe0 con 0x7f05540cd4d0 2023-07-08T12:39:37.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.776+0000 7f0552ffd700 1 --2- 172.21.15.12:0/2691523348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05540cd4d0 0x7f05540ce2e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f05540721c0 tx=0x7f053c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f7d9ba221174c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:37.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.776+0000 7f05527fc700 1 -- 172.21.15.12:0/2691523348 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c004230 con 0x7f05540cd4d0 2023-07-08T12:39:37.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.776+0000 7f05527fc700 1 -- 172.21.15.12:0/2691523348 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c004390 con 0x7f05540cd4d0 2023-07-08T12:39:37.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.776+0000 7f05527fc700 1 -- 172.21.15.12:0/2691523348 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c0036a0 con 0x7f05540cd4d0 2023-07-08T12:39:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.777+0000 7f05589a0700 1 -- 172.21.15.12:0/2691523348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05540cd4d0 msgr2=0x7f05540ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.777+0000 7f05589a0700 1 --2- 172.21.15.12:0/2691523348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05540cd4d0 0x7f05540ce2e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f05540721c0 tx=0x7f053c009bd0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.777+0000 7f05589a0700 1 -- 172.21.15.12:0/2691523348 shutdown_connections 2023-07-08T12:39:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.777+0000 7f05589a0700 1 --2- 172.21.15.12:0/2691523348 >> v2:172.21.15.160:3300/0 conn(0x7f05540b5f90 0x7f05540cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.777+0000 7f05589a0700 1 --2- 172.21.15.12:0/2691523348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05540cd4d0 0x7f05540ce2e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.777+0000 7f05589a0700 1 -- 172.21.15.12:0/2691523348 >> 172.21.15.12:0/2691523348 conn(0x7f055406e3e0 msgr2=0x7f055406e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:37.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.777+0000 7f05589a0700 1 -- 172.21.15.12:0/2691523348 shutdown_connections 2023-07-08T12:39:37.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.777+0000 7f05589a0700 1 -- 172.21.15.12:0/2691523348 wait complete. 2023-07-08T12:39:37.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.778+0000 7f05589a0700 1 Processor -- start 2023-07-08T12:39:37.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.778+0000 7f05589a0700 1 -- start start 2023-07-08T12:39:37.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f05589a0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05540b5f90 0x7f05540c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f05589a0700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05540cd4d0 0x7f0554157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f05589a0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05540ce820 0x7f05541577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f05589a0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05540c6a50 con 0x7f05540ce820 2023-07-08T12:39:37.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f05589a0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05540c77e0 con 0x7f05540b5f90 2023-07-08T12:39:37.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f05589a0700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0554156300 con 0x7f05540cd4d0 2023-07-08T12:39:37.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f0553fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05540ce820 0x7f05541577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f0553fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05540ce820 0x7f05541577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51886/0 (socket says 172.21.15.12:51886) 2023-07-08T12:39:37.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f0553fff700 1 -- 172.21.15.12:0/2634779282 learned_addr learned my addr 172.21.15.12:0/2634779282 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:37.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f05537fe700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05540b5f90 0x7f05540c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f0552ffd700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05540cd4d0 0x7f0554157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f0553fff700 1 -- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05540cd4d0 msgr2=0x7f0554157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f0553fff700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05540cd4d0 0x7f0554157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f0553fff700 1 -- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05540b5f90 msgr2=0x7f05540c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f0553fff700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05540b5f90 0x7f05540c6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f0553fff700 1 -- 172.21.15.12:0/2634779282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c009550 con 0x7f05540ce820 2023-07-08T12:39:37.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.779+0000 7f0553fff700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05540ce820 0x7f05541577a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f054c00cc10 tx=0x7f054c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:37.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.780+0000 7f054bfff700 1 -- 172.21.15.12:0/2634779282 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c00aa60 con 0x7f05540ce820 2023-07-08T12:39:37.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.780+0000 7f054bfff700 1 -- 172.21.15.12:0/2634779282 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c00abc0 con 0x7f05540ce820 2023-07-08T12:39:37.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.780+0000 7f05589a0700 1 -- 172.21.15.12:0/2634779282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05541565e0 con 0x7f05540ce820 2023-07-08T12:39:37.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.780+0000 7f054bfff700 1 -- 172.21.15.12:0/2634779282 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c012420 con 0x7f05540ce820 2023-07-08T12:39:37.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.780+0000 7f05589a0700 1 -- 172.21.15.12:0/2634779282 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0554156ba0 con 0x7f05540ce820 2023-07-08T12:39:37.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.781+0000 7f054bfff700 1 -- 172.21.15.12:0/2634779282 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f054c012580 con 0x7f05540ce820 2023-07-08T12:39:37.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.782+0000 7f054bfff700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f053406ace0 0x7f053406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.782+0000 7f054bfff700 1 -- 172.21.15.12:0/2634779282 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f054c089970 con 0x7f05540ce820 2023-07-08T12:39:37.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.782+0000 7f05537fe700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f053406ace0 0x7f053406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.782+0000 7f05537fe700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f053406ace0 0x7f053406d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0544006fd0 tx=0x7f0544008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:37.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.784+0000 7f0565d87b80 1 Processor -- start 2023-07-08T12:39:37.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.784+0000 7f0565d87b80 1 -- start start 2023-07-08T12:39:37.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.784+0000 7f0565d87b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 0x56352a9e7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.784+0000 7f0565d87b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a99e900 0x56352a9cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.784+0000 7f0565d87b80 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9e6970 0x56352a9cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.784+0000 7f0565d87b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352a8a5c60 con 0x56352a99e900 2023-07-08T12:39:37.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.784+0000 7f0565d87b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352a8f89f0 con 0x56352a9e65a0 2023-07-08T12:39:37.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f0565d87b80 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x56352a8f1e80 con 0x56352a9e6970 2023-07-08T12:39:37.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f0552ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a99e900 0x56352a9cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f0552ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a99e900 0x56352a9cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51892/0 (socket says 172.21.15.12:51892) 2023-07-08T12:39:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f0552ffd700 1 -- 172.21.15.12:0/3751267993 learned_addr learned my addr 172.21.15.12:0/3751267993 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f05537fe700 1 --2- 172.21.15.12:0/3751267993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 0x56352a9e7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f0553fff700 1 --2- 172.21.15.12:0/3751267993 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9e6970 0x56352a9cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f0552ffd700 1 -- 172.21.15.12:0/3751267993 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9e6970 msgr2=0x56352a9cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f0552ffd700 1 --2- 172.21.15.12:0/3751267993 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9e6970 0x56352a9cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f0552ffd700 1 -- 172.21.15.12:0/3751267993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 msgr2=0x56352a9e7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f0552ffd700 1 --2- 172.21.15.12:0/3751267993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 0x56352a9e7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f0552ffd700 1 -- 172.21.15.12:0/3751267993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56352a71fd90 con 0x56352a99e900 2023-07-08T12:39:37.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f0552ffd700 1 --2- 172.21.15.12:0/3751267993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a99e900 0x56352a9cb240 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f053c004000 tx=0x7f053c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb152019838bb4ac server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:37.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f05497fa700 1 -- 172.21.15.12:0/3751267993 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c004230 con 0x56352a99e900 2023-07-08T12:39:37.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f05497fa700 1 -- 172.21.15.12:0/3751267993 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c003850 con 0x56352a99e900 2023-07-08T12:39:37.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.785+0000 7f05497fa700 1 -- 172.21.15.12:0/3751267993 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c003bc0 con 0x56352a99e900 2023-07-08T12:39:37.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.786+0000 7f0565d87b80 1 -- 172.21.15.12:0/3751267993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a99e900 msgr2=0x56352a9cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.786+0000 7f0565d87b80 1 --2- 172.21.15.12:0/3751267993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a99e900 0x56352a9cb240 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f053c004000 tx=0x7f053c010000 comp rx=0 tx=0).stop 2023-07-08T12:39:37.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.786+0000 7f0565d87b80 1 -- 172.21.15.12:0/3751267993 shutdown_connections 2023-07-08T12:39:37.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.786+0000 7f0565d87b80 1 --2- 172.21.15.12:0/3751267993 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9e6970 0x56352a9cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.786+0000 7f0565d87b80 1 --2- 172.21.15.12:0/3751267993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 0x56352a9e7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.786+0000 7f0565d87b80 1 --2- 172.21.15.12:0/3751267993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a99e900 0x56352a9cb240 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.786+0000 7f0565d87b80 1 -- 172.21.15.12:0/3751267993 >> 172.21.15.12:0/3751267993 conn(0x56352a995ce0 msgr2=0x56352a99b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:37.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.786+0000 7f0565d87b80 1 -- 172.21.15.12:0/3751267993 shutdown_connections 2023-07-08T12:39:37.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.786+0000 7f0565d87b80 1 -- 172.21.15.12:0/3751267993 wait complete. 2023-07-08T12:39:37.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.787+0000 7f0565d87b80 1 -- start start 2023-07-08T12:39:37.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.787+0000 7f0565d87b80 1 Processor -- start 2023-07-08T12:39:37.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.787+0000 7f0565d87b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 0x56352a9c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.787+0000 7f0565d87b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e6970 0x56352a9c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.787+0000 7f0565d87b80 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9c2e60 0x56352a9b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.787+0000 7f0565d87b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352a891a70 con 0x56352a9e6970 2023-07-08T12:39:37.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.787+0000 7f0565d87b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352a891770 con 0x56352a9e65a0 2023-07-08T12:39:37.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.787+0000 7f0565d87b80 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x56352a7758d0 con 0x56352a9c2e60 2023-07-08T12:39:37.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0565d87b80 10 client.0 _ll_register_callbacks cb 0x56352a88f710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-08T12:39:37.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0552ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e6970 0x56352a9c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0552ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e6970 0x56352a9c39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51898/0 (socket says 172.21.15.12:51898) 2023-07-08T12:39:37.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0552ffd700 1 -- 172.21.15.12:0/2483781349 learned_addr learned my addr 172.21.15.12:0/2483781349 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:37.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f05537fe700 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 0x56352a9c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0553fff700 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9c2e60 0x56352a9b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0552ffd700 1 -- 172.21.15.12:0/2483781349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9c2e60 msgr2=0x56352a9b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:37.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0552ffd700 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9c2e60 0x56352a9b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0552ffd700 1 -- 172.21.15.12:0/2483781349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 msgr2=0x56352a9c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:37.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0552ffd700 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 0x56352a9c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0552ffd700 1 -- 172.21.15.12:0/2483781349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05540cffe0 con 0x56352a9e6970 2023-07-08T12:39:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0553fff700 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9c2e60 0x56352a9b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:39:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f05327fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-07-08T12:39:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0552ffd700 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e6970 0x56352a9c39f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f053c009fd0 tx=0x7f053c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9098feec16e679d server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f05327fc700 1 -- 172.21.15.12:0/2483781349 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c0039c0 con 0x56352a9e6970 2023-07-08T12:39:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f05327fc700 1 -- 172.21.15.12:0/2483781349 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f053c003b20 con 0x56352a9e6970 2023-07-08T12:39:37.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0565d87b80 1 -- 172.21.15.12:0/2483781349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56352a9932f0 con 0x56352a9e6970 2023-07-08T12:39:37.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f05327fc700 1 -- 172.21.15.12:0/2483781349 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f053c003ce0 con 0x56352a9e6970 2023-07-08T12:39:37.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0565d87b80 10 client.5529 fetch_fsmap learned FSMap version 17 2023-07-08T12:39:37.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.788+0000 7f0565d87b80 1 -- 172.21.15.12:0/2483781349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x56352a708c10 con 0x56352a9e6970 2023-07-08T12:39:37.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.789+0000 7f05327fc700 1 -- 172.21.15.12:0/2483781349 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56352a9932f0 con 0x56352a9e6970 2023-07-08T12:39:37.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.789+0000 7f05327fc700 1 -- 172.21.15.12:0/2483781349 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f053c018070 con 0x56352a9e6970 2023-07-08T12:39:37.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.789+0000 7f0565d87b80 10 client.5529 fetch_fsmap finished waiting for FSMap version 17 2023-07-08T12:39:37.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.789+0000 7f0565d87b80 10 client.5529 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-08T12:39:37.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.789+0000 7f0565d87b80 20 client.5529 populate_metadata read hostname 'smithi012' 2023-07-08T12:39:37.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.789+0000 7f0565d87b80 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.12:6833/590221748,v1:172.21.15.12:6835/590221748] conn(0x56352a9c5060 0x56352a9c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.789+0000 7f0565d87b80 4 client.5529 mds_command: new command op to 5346 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-08T12:39:37.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.789+0000 7f0565d87b80 1 -- 172.21.15.12:0/2483781349 --> [v2:172.21.15.12:6833/590221748,v1:172.21.15.12:6835/590221748] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56352a8eae60 con 0x56352a9c5060 2023-07-08T12:39:37.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.789+0000 7f05537fe700 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.12:6833/590221748,v1:172.21.15.12:6835/590221748] conn(0x56352a9c5060 0x56352a9c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.789+0000 7f05327fc700 10 client.5529 ms_handle_connect on v2:172.21.15.12:6833/590221748 2023-07-08T12:39:37.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.789+0000 7f05537fe700 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.12:6833/590221748,v1:172.21.15.12:6835/590221748] conn(0x56352a9c5060 0x56352a9c5430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476c8728ae8a1b11 server_cookie=b1d6d6d4580b58f in_seq=0 out_seq=0 2023-07-08T12:39:37.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f05327fc700 1 -- 172.21.15.12:0/2483781349 <== mds.0 v2:172.21.15.12:6833/590221748 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56352a8eae60 con 0x56352a9c5060 2023-07-08T12:39:37.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f05327fc700 10 client.5529 handle_command_reply: tid=0 2023-07-08T12:39:37.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f0565d87b80 1 client.5529 shutdown 2023-07-08T12:39:37.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f0565d87b80 1 -- 172.21.15.12:0/2483781349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e6970 msgr2=0x56352a9c39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f0565d87b80 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e6970 0x56352a9c39f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f053c009fd0 tx=0x7f053c0047c0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f0565d87b80 1 -- 172.21.15.12:0/2483781349 shutdown_connections 2023-07-08T12:39:37.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f0565d87b80 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.12:6833/590221748,v1:172.21.15.12:6835/590221748] conn(0x56352a9c5060 0x56352a9c5430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f0565d87b80 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9c2e60 0x56352a9b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f0565d87b80 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 0x56352a9c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f05327fc700 0 client.5529 ms_handle_reset on v2:172.21.15.12:6833/590221748 2023-07-08T12:39:37.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f0565d87b80 1 --2- 172.21.15.12:0/2483781349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e6970 0x56352a9c39f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f0565d87b80 1 -- 172.21.15.12:0/2483781349 >> 172.21.15.12:0/2483781349 conn(0x56352a995ce0 msgr2=0x56352a99d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:37.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f0565d87b80 1 -- 172.21.15.12:0/2483781349 shutdown_connections 2023-07-08T12:39:37.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f0565d87b80 1 -- 172.21.15.12:0/2483781349 wait complete. 2023-07-08T12:39:37.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.790+0000 7f0565d87b80 20 client.5529 trim_cache size 0 max 16384 2023-07-08T12:39:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.808+0000 7f0565d87b80 1 Processor -- start 2023-07-08T12:39:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.808+0000 7f0565d87b80 1 -- start start 2023-07-08T12:39:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f0565d87b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9c2e60 0x56352a9e3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f0565d87b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 0x56352a9e4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f0565d87b80 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9e6970 0x56352a99b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f0565d87b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352a99e6f0 con 0x56352a9c2e60 2023-07-08T12:39:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f0565d87b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352a99ef10 con 0x56352a9e65a0 2023-07-08T12:39:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f0565d87b80 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x56352a9e3600 con 0x56352a9e6970 2023-07-08T12:39:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f05537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9c2e60 0x56352a9e3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f05537fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9c2e60 0x56352a9e3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51904/0 (socket says 172.21.15.12:51904) 2023-07-08T12:39:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f05537fe700 1 -- 172.21.15.12:0/2494303825 learned_addr learned my addr 172.21.15.12:0/2494303825 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f05537fe700 1 -- 172.21.15.12:0/2494303825 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9e6970 msgr2=0x56352a99b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f05537fe700 1 --2- 172.21.15.12:0/2494303825 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9e6970 0x56352a99b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f05537fe700 1 -- 172.21.15.12:0/2494303825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 msgr2=0x56352a9e4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f05537fe700 1 --2- 172.21.15.12:0/2494303825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 0x56352a9e4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f05537fe700 1 -- 172.21.15.12:0/2494303825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56352a746200 con 0x56352a9c2e60 2023-07-08T12:39:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f05537fe700 1 --2- 172.21.15.12:0/2494303825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9c2e60 0x56352a9e3a40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x56352a99caf0 tx=0x7f0544016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8865ae9575f3296 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.809+0000 7f0549ffb700 1 -- 172.21.15.12:0/2494303825 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05440045a0 con 0x56352a9c2e60 2023-07-08T12:39:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.810+0000 7f0549ffb700 1 -- 172.21.15.12:0/2494303825 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054400c040 con 0x56352a9c2e60 2023-07-08T12:39:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.810+0000 7f0549ffb700 1 -- 172.21.15.12:0/2494303825 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544029520 con 0x56352a9c2e60 2023-07-08T12:39:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.810+0000 7f0565d87b80 1 -- 172.21.15.12:0/2494303825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9c2e60 msgr2=0x56352a9e3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.810+0000 7f0565d87b80 1 --2- 172.21.15.12:0/2494303825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9c2e60 0x56352a9e3a40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x56352a99caf0 tx=0x7f0544016000 comp rx=0 tx=0).stop 2023-07-08T12:39:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.810+0000 7f0565d87b80 1 -- 172.21.15.12:0/2494303825 shutdown_connections 2023-07-08T12:39:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.810+0000 7f0565d87b80 1 --2- 172.21.15.12:0/2494303825 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9e6970 0x56352a99b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.810+0000 7f0565d87b80 1 --2- 172.21.15.12:0/2494303825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9c2e60 0x56352a9e3a40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.810+0000 7f0565d87b80 1 --2- 172.21.15.12:0/2494303825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9e65a0 0x56352a9e4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.810+0000 7f0565d87b80 1 -- 172.21.15.12:0/2494303825 >> 172.21.15.12:0/2494303825 conn(0x56352a995ce0 msgr2=0x56352a9c3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.810+0000 7f0565d87b80 1 -- 172.21.15.12:0/2494303825 shutdown_connections 2023-07-08T12:39:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.810+0000 7f0565d87b80 1 -- 172.21.15.12:0/2494303825 wait complete. 2023-07-08T12:39:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.811+0000 7f0565d87b80 1 -- start start 2023-07-08T12:39:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.811+0000 7f0565d87b80 1 Processor -- start 2023-07-08T12:39:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0565d87b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9c2e60 0x56352a9c7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0565d87b80 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e65a0 0x56352a9e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0565d87b80 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9e6970 0x56352a9e4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0565d87b80 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352aa06740 con 0x56352a9e65a0 2023-07-08T12:39:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0565d87b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352a9c7960 con 0x56352a9c2e60 2023-07-08T12:39:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0565d87b80 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x56352aa38bb0 con 0x56352a9e6970 2023-07-08T12:39:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0565d87b80 10 client.0 _ll_register_callbacks cb 0x7f053c006d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-08T12:39:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0552ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e65a0 0x56352a9e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0552ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e65a0 0x56352a9e3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51912/0 (socket says 172.21.15.12:51912) 2023-07-08T12:39:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0552ffd700 1 -- 172.21.15.12:0/1501098811 learned_addr learned my addr 172.21.15.12:0/1501098811 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0552ffd700 1 -- 172.21.15.12:0/1501098811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9e6970 msgr2=0x56352a9e4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0552ffd700 1 --2- 172.21.15.12:0/1501098811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9e6970 0x56352a9e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0552ffd700 1 -- 172.21.15.12:0/1501098811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9c2e60 msgr2=0x56352a9c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0552ffd700 1 --2- 172.21.15.12:0/1501098811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9c2e60 0x56352a9c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0552ffd700 1 -- 172.21.15.12:0/1501098811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054400fcd0 con 0x56352a9e65a0 2023-07-08T12:39:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0552ffd700 1 --2- 172.21.15.12:0/1501098811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e65a0 0x56352a9e3fc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f053c009130 tx=0x7f053c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ccd0df26177a9e server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f05327fc700 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-07-08T12:39:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f05327fc700 1 -- 172.21.15.12:0/1501098811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c004230 con 0x56352a9e65a0 2023-07-08T12:39:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f05327fc700 1 -- 172.21.15.12:0/1501098811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f053c000e40 con 0x56352a9e65a0 2023-07-08T12:39:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0565d87b80 1 -- 172.21.15.12:0/1501098811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f053c003850 con 0x56352a9e65a0 2023-07-08T12:39:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f05327fc700 1 -- 172.21.15.12:0/1501098811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f053c003bc0 con 0x56352a9e65a0 2023-07-08T12:39:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.812+0000 7f0565d87b80 10 client.5535 fetch_fsmap learned FSMap version 17 2023-07-08T12:39:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f0565d87b80 1 -- 172.21.15.12:0/1501098811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x56352a71e260 con 0x56352a9e65a0 2023-07-08T12:39:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f05327fc700 1 -- 172.21.15.12:0/1501098811 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c003850 con 0x56352a9e65a0 2023-07-08T12:39:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f05327fc700 1 -- 172.21.15.12:0/1501098811 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f053c018070 con 0x56352a9e65a0 2023-07-08T12:39:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f0565d87b80 10 client.5535 fetch_fsmap finished waiting for FSMap version 17 2023-07-08T12:39:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f0565d87b80 10 client.5535 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-08T12:39:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f0565d87b80 20 client.5535 populate_metadata read hostname 'smithi012' 2023-07-08T12:39:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f0565d87b80 1 --2- 172.21.15.12:0/1501098811 >> [v2:172.21.15.12:6833/590221748,v1:172.21.15.12:6835/590221748] conn(0x56352aa37d00 0x56352aa380d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f0565d87b80 4 client.5535 mds_command: new command op to 5346 tid=0[{"prefix": "status"}] 2023-07-08T12:39:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f0565d87b80 1 -- 172.21.15.12:0/1501098811 --> [v2:172.21.15.12:6833/590221748,v1:172.21.15.12:6835/590221748] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56352a8ad000 con 0x56352aa37d00 2023-07-08T12:39:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f05537fe700 1 --2- 172.21.15.12:0/1501098811 >> [v2:172.21.15.12:6833/590221748,v1:172.21.15.12:6835/590221748] conn(0x56352aa37d00 0x56352aa380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f05327fc700 10 client.5535 ms_handle_connect on v2:172.21.15.12:6833/590221748 2023-07-08T12:39:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f05537fe700 1 --2- 172.21.15.12:0/1501098811 >> [v2:172.21.15.12:6833/590221748,v1:172.21.15.12:6835/590221748] conn(0x56352aa37d00 0x56352aa380d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63144ad5062a4e72 server_cookie=fc0c4b68a6285c07 in_seq=0 out_seq=0 2023-07-08T12:39:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f05327fc700 1 -- 172.21.15.12:0/1501098811 <== mds.0 v2:172.21.15.12:6833/590221748 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x56352a8ad000 con 0x56352aa37d00 2023-07-08T12:39:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f05327fc700 10 client.5535 handle_command_reply: tid=0 2023-07-08T12:39:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.813+0000 7f0565d87b80 1 client.5535 shutdown 2023-07-08T12:39:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f0565d87b80 1 -- 172.21.15.12:0/1501098811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e65a0 msgr2=0x56352a9e3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f0565d87b80 1 --2- 172.21.15.12:0/1501098811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e65a0 0x56352a9e3fc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f053c009130 tx=0x7f053c004750 comp rx=0 tx=0).stop 2023-07-08T12:39:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f0565d87b80 1 -- 172.21.15.12:0/1501098811 shutdown_connections 2023-07-08T12:39:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f0565d87b80 1 --2- 172.21.15.12:0/1501098811 >> [v2:172.21.15.12:6833/590221748,v1:172.21.15.12:6835/590221748] conn(0x56352aa37d00 0x56352aa380d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f0565d87b80 1 --2- 172.21.15.12:0/1501098811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56352a9e6970 0x56352a9e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f05327fc700 0 client.5535 ms_handle_reset on v2:172.21.15.12:6833/590221748 2023-07-08T12:39:37.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f0565d87b80 1 --2- 172.21.15.12:0/1501098811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56352a9c2e60 0x56352a9c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f0565d87b80 1 --2- 172.21.15.12:0/1501098811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56352a9e65a0 0x56352a9e3fc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f0565d87b80 1 -- 172.21.15.12:0/1501098811 >> 172.21.15.12:0/1501098811 conn(0x56352a995ce0 msgr2=0x56352a9c2040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:37.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f0565d87b80 1 -- 172.21.15.12:0/1501098811 shutdown_connections 2023-07-08T12:39:37.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f0565d87b80 1 -- 172.21.15.12:0/1501098811 wait complete. 2023-07-08T12:39:37.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f0565d87b80 20 client.5535 trim_cache size 0 max 16384 2023-07-08T12:39:37.826 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-07-08T12:39:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "71d270f5-58a7-4223-8d07-37436121729a", 2023-07-08T12:39:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2023-07-08T12:39:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 5346, 2023-07-08T12:39:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2023-07-08T12:39:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2023-07-08T12:39:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2023-07-08T12:39:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 5.998193637, 2023-07-08T12:39:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 17, 2023-07-08T12:39:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 42, 2023-07-08T12:39:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 42, 2023-07-08T12:39:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 13.989702282 2023-07-08T12:39:37.827 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-07-08T12:39:37.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f05589a0700 1 -- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f053406ace0 msgr2=0x7f053406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f05589a0700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f053406ace0 0x7f053406d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0544006fd0 tx=0x7f0544008040 comp rx=0 tx=0).stop 2023-07-08T12:39:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f05589a0700 1 -- 172.21.15.12:0/2634779282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05540ce820 msgr2=0x7f05541577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f05589a0700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05540ce820 0x7f05541577a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f054c00cc10 tx=0x7f054c00cf20 comp rx=0 tx=0).stop 2023-07-08T12:39:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f05589a0700 1 -- 172.21.15.12:0/2634779282 shutdown_connections 2023-07-08T12:39:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f05589a0700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f053406ace0 0x7f053406d190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f05589a0700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05540ce820 0x7f05541577a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f05589a0700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05540b5f90 0x7f05540c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f05589a0700 1 --2- 172.21.15.12:0/2634779282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05540cd4d0 0x7f0554157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f05589a0700 1 -- 172.21.15.12:0/2634779282 >> 172.21.15.12:0/2634779282 conn(0x7f055406e3e0 msgr2=0x7f05540ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:37.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f05589a0700 1 -- 172.21.15.12:0/2634779282 shutdown_connections 2023-07-08T12:39:37.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.814+0000 7f05589a0700 1 -- 172.21.15.12:0/2634779282 wait complete. 2023-07-08T12:39:37.830 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-08T12:39:37.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.953+0000 7f929244e700 1 Processor -- start 2023-07-08T12:39:37.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.953+0000 7f929244e700 1 -- start start 2023-07-08T12:39:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.954+0000 7f929244e700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f928c159c30 0x7f928c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.954+0000 7f929244e700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928c15c890 con 0x7f928c160790 2023-07-08T12:39:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.954+0000 7f929244e700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928c06f620 con 0x7f928c069f30 2023-07-08T12:39:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.954+0000 7f929244e700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928c15c730 con 0x7f928c159c30 2023-07-08T12:39:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.954+0000 7f928b7fe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f928c160790 0x7f928c06ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39316/0 (socket says 172.21.15.12:39316) 2023-07-08T12:39:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.954+0000 7f928b7fe700 1 -- 172.21.15.12:0/874170188 learned_addr learned my addr 172.21.15.12:0/874170188 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.954+0000 7f928bfff700 1 --2- 172.21.15.12:0/874170188 >> v2:172.21.15.160:3300/0 conn(0x7f928c159c30 0x7f928c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.954+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966026035 0 0) 0x7f928c15c890 con 0x7f928c160790 2023-07-08T12:39:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.954+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92780035b0 con 0x7f928c160790 2023-07-08T12:39:37.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273269409 0 0) 0x7f928c06f620 con 0x7f928c069f30 2023-07-08T12:39:37.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f928c15c890 con 0x7f928c069f30 2023-07-08T12:39:37.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4085618946 0 0) 0x7f92780035b0 con 0x7f928c160790 2023-07-08T12:39:37.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 >> v2:172.21.15.160:3300/0 conn(0x7f928c159c30 msgr2=0x7f928c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:37.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 --2- 172.21.15.12:0/874170188 >> v2:172.21.15.160:3300/0 conn(0x7f928c159c30 0x7f928c0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 >> v1:172.21.15.160:6789/0 conn(0x7f928c069f30 legacy=0x7f928c15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c160c90 con 0x7f928c160790 2023-07-08T12:39:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f927c001670 con 0x7f928c160790 2023-07-08T12:39:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 >> v1:172.21.15.12:6789/0 conn(0x7f928c160790 legacy=0x7f928c06ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 --2- 172.21.15.12:0/874170188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9278003b90 0x7f9278003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 --2- 172.21.15.12:0/874170188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9278004850 0x7f9278004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 --2- 172.21.15.12:0/874170188 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9278005200 0x7f927800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c15c730 con 0x7f9278004850 2023-07-08T12:39:37.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927800c060 con 0x7f9278003b90 2023-07-08T12:39:37.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927800c220 con 0x7f9278005200 2023-07-08T12:39:37.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928b7fe700 1 --2- 172.21.15.12:0/874170188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9278004850 0x7f9278004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928bfff700 1 --2- 172.21.15.12:0/874170188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9278003b90 0x7f9278003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f92909eb700 1 --2- 172.21.15.12:0/874170188 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9278005200 0x7f927800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928b7fe700 1 -- 172.21.15.12:0/874170188 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9278005200 msgr2=0x7f927800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:37.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928b7fe700 1 --2- 172.21.15.12:0/874170188 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9278005200 0x7f927800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928b7fe700 1 -- 172.21.15.12:0/874170188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9278003b90 msgr2=0x7f9278003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:37.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928b7fe700 1 --2- 172.21.15.12:0/874170188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9278003b90 0x7f9278003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928b7fe700 1 -- 172.21.15.12:0/874170188 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92780035b0 con 0x7f9278004850 2023-07-08T12:39:37.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f92909eb700 1 -- 172.21.15.12:0/874170188 reap_dead start 2023-07-08T12:39:37.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928b7fe700 1 --2- 172.21.15.12:0/874170188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9278004850 0x7f9278004cc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f928c160bc0 tx=0x7f9278005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3b594fc5dd18ff server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:37.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00cd80 con 0x7f9278004850 2023-07-08T12:39:37.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c001af0 con 0x7f9278004850 2023-07-08T12:39:37.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.955+0000 7f928affd700 1 -- 172.21.15.12:0/874170188 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c005ba0 con 0x7f9278004850 2023-07-08T12:39:37.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.956+0000 7f929244e700 1 -- 172.21.15.12:0/874170188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9278004850 msgr2=0x7f9278004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.956+0000 7f929244e700 1 --2- 172.21.15.12:0/874170188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9278004850 0x7f9278004cc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f928c160bc0 tx=0x7f9278005880 comp rx=0 tx=0).stop 2023-07-08T12:39:37.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.956+0000 7f929244e700 1 -- 172.21.15.12:0/874170188 shutdown_connections 2023-07-08T12:39:37.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.956+0000 7f929244e700 1 --2- 172.21.15.12:0/874170188 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9278004850 0x7f9278004cc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.956+0000 7f929244e700 1 -- 172.21.15.12:0/874170188 >> 172.21.15.12:0/874170188 conn(0x7f928c0cc290 msgr2=0x7f928c0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:37.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.956+0000 7f929244e700 1 -- 172.21.15.12:0/874170188 shutdown_connections 2023-07-08T12:39:37.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.956+0000 7f929244e700 1 -- 172.21.15.12:0/874170188 wait complete. 2023-07-08T12:39:37.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.957+0000 7f929244e700 1 Processor -- start 2023-07-08T12:39:37.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.957+0000 7f929244e700 1 -- start start 2023-07-08T12:39:37.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.958+0000 7f929244e700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9278004850 0x7f928c1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.958+0000 7f929244e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f928c157020 0x7f928c0af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.958+0000 7f929244e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f928c15a000 0x7f928c15a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.958+0000 7f929244e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c15a9e0 con 0x7f928c157020 2023-07-08T12:39:37.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.958+0000 7f929244e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c0afcb0 con 0x7f928c15a000 2023-07-08T12:39:37.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.958+0000 7f929244e700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928c0aff90 con 0x7f9278004850 2023-07-08T12:39:37.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.958+0000 7f928b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f928c157020 0x7f928c0af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.958+0000 7f928b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f928c157020 0x7f928c0af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51932/0 (socket says 172.21.15.12:51932) 2023-07-08T12:39:37.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.958+0000 7f928b7fe700 1 -- 172.21.15.12:0/2149587211 learned_addr learned my addr 172.21.15.12:0/2149587211 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:37.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.958+0000 7f928bfff700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9278004850 0x7f928c1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.958+0000 7f92909eb700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f928c15a000 0x7f928c15a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.959+0000 7f928b7fe700 1 -- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9278004850 msgr2=0x7f928c1561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.959+0000 7f928b7fe700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9278004850 0x7f928c1561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.959+0000 7f928b7fe700 1 -- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f928c15a000 msgr2=0x7f928c15a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:37.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.959+0000 7f928b7fe700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f928c15a000 0x7f928c15a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:37.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.959+0000 7f928b7fe700 1 -- 172.21.15.12:0/2149587211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c0b0270 con 0x7f928c157020 2023-07-08T12:39:37.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.959+0000 7f928b7fe700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f928c157020 0x7f928c0af270 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f927800c030 tx=0x7f927c001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:37.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.959+0000 7f9288ff9700 1 -- 172.21.15.12:0/2149587211 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c0072d0 con 0x7f928c157020 2023-07-08T12:39:37.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.959+0000 7f9288ff9700 1 -- 172.21.15.12:0/2149587211 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c007450 con 0x7f928c157020 2023-07-08T12:39:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.959+0000 7f929244e700 1 -- 172.21.15.12:0/2149587211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928c0c4630 con 0x7f928c157020 2023-07-08T12:39:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.959+0000 7f9288ff9700 1 -- 172.21.15.12:0/2149587211 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c007800 con 0x7f928c157020 2023-07-08T12:39:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.959+0000 7f929244e700 1 -- 172.21.15.12:0/2149587211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928c0b2640 con 0x7f928c157020 2023-07-08T12:39:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.960+0000 7f929244e700 1 -- 172.21.15.12:0/2149587211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928c04ea60 con 0x7f928c157020 2023-07-08T12:39:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.960+0000 7f9288ff9700 1 -- 172.21.15.12:0/2149587211 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f927c01d420 con 0x7f928c157020 2023-07-08T12:39:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.961+0000 7f9288ff9700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f926c06ace0 0x7f926c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.961+0000 7f9288ff9700 1 -- 172.21.15.12:0/2149587211 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f927c089a30 con 0x7f928c157020 2023-07-08T12:39:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.961+0000 7f928bfff700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f926c06ace0 0x7f926c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.962+0000 7f928bfff700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f926c06ace0 0x7f926c06d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f928c0d0650 tx=0x7f9274009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:37.966+0000 7f9288ff9700 1 -- 172.21.15.12:0/2149587211 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f927c054af0 con 0x7f928c157020 2023-07-08T12:39:38.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.208+0000 7f929244e700 1 -- 172.21.15.12:0/2149587211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f928c0b5790 con 0x7f928c157020 2023-07-08T12:39:38.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.208+0000 7f9288ff9700 1 -- 172.21.15.12:0/2149587211 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f927c058110 con 0x7f928c157020 2023-07-08T12:39:38.211 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-08T12:39:38.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.210+0000 7f929244e700 1 -- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f926c06ace0 msgr2=0x7f926c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:38.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.210+0000 7f929244e700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f926c06ace0 0x7f926c06d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f928c0d0650 tx=0x7f9274009210 comp rx=0 tx=0).stop 2023-07-08T12:39:38.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.210+0000 7f929244e700 1 -- 172.21.15.12:0/2149587211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f928c157020 msgr2=0x7f928c0af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.210+0000 7f929244e700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f928c157020 0x7f928c0af270 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f927800c030 tx=0x7f927c001980 comp rx=0 tx=0).stop 2023-07-08T12:39:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.210+0000 7f929244e700 1 -- 172.21.15.12:0/2149587211 shutdown_connections 2023-07-08T12:39:38.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.210+0000 7f929244e700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f926c06ace0 0x7f926c06d190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.210+0000 7f929244e700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f928c15a000 0x7f928c15a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.210+0000 7f929244e700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9278004850 0x7f928c1561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.210+0000 7f929244e700 1 --2- 172.21.15.12:0/2149587211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f928c157020 0x7f928c0af270 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.210+0000 7f929244e700 1 -- 172.21.15.12:0/2149587211 >> 172.21.15.12:0/2149587211 conn(0x7f928c0cc290 msgr2=0x7f928c15b580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.210+0000 7f929244e700 1 -- 172.21.15.12:0/2149587211 shutdown_connections 2023-07-08T12:39:38.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:38.210+0000 7f929244e700 1 -- 172.21.15.12:0/2149587211 wait complete. 2023-07-08T12:39:38.224 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-08T12:39:38.225 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-08T12:39:38.225 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2023-07-08T12:39:38.225 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-08T12:39:38.225 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-08T12:39:38.226 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-08T12:39:38.226 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-07-08T12:39:38.226 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-08T12:39:38.226 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-08T12:39:38.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:38.228 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2023-07-08T12:39:38.246 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-08T12:39:38.246 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-08T12:39:38.246 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2023-07-08T12:39:38.247 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:39:38.247 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2023-07-08T12:39:38.247 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:39:38.247 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:39:38.247 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2023-07-08T12:39:38.247 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:39:38.247 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2023-07-08T12:39:38.248 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-08T12:39:38.248 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2023-07-08T12:39:38.248 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-08T12:39:38.248 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42071sec preferred_lft 42071sec 2023-07-08T12:39:38.248 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2023-07-08T12:39:38.248 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:39:38.248 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-08T12:39:38.249 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2023-07-08T12:39:38.249 INFO:teuthology.orchestra.run.smithi012.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-08T12:39:38.249 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether d6:f4:33:eb:94:9f brd ff:ff:ff:ff:ff:ff 2023-07-08T12:39:38.249 INFO:teuthology.orchestra.run.smithi012.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-08T12:39:38.249 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:39:38.249 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::84cf:ecff:fe20:a70e/64 scope link 2023-07-08T12:39:38.249 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:39:38.249 INFO:teuthology.orchestra.run.smithi012.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-08T12:39:38.250 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether d6:f4:33:eb:94:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-08T12:39:38.250 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::d4f4:33ff:feeb:949f/64 scope link 2023-07-08T12:39:38.250 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:39:38.250 INFO:teuthology.orchestra.run.smithi012.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-08T12:39:38.250 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether fe:b8:cf:16:60:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-08T12:39:38.250 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::fcb8:cfff:fe16:60d1/64 scope link 2023-07-08T12:39:38.250 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-07-08T12:39:38.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:38.251 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-08T12:39:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:1 2023-07-08T12:39:38.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:38.313 DEBUG:teuthology.orchestra.run.smithi012:> route 2023-07-08T12:39:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2023-07-08T12:39:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-08T12:39:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-08T12:39:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-08T12:39:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-08T12:39:38.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:38.373 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-07-08T12:39:38.374 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-07-08T12:39:38.374 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-08T12:39:38.374 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-08T12:39:38.375 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-08T12:39:38.375 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-07-08T12:39:38.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:38.479 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2023-07-08T12:39:38.495 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-08T12:39:38.496 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-08T12:39:38.496 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-08T12:39:38.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:38.497 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2023-07-08T12:39:38.563 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-08T12:39:38.564 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-08T12:39:38.620 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-08T12:39:38.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-08T12:39:38.620 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-08T12:39:38.676 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-07-08T12:39:38.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:38.677 DEBUG:teuthology.orchestra.run.smithi012:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-08T12:39:38.763 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-08T12:39:38.764 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-07-08T12:39:38.764 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-07-08T12:39:38.772 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-08T12:39:38.773 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-07-08T12:39:38.773 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-07-08T12:39:38.773 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-07-08T12:39:38.773 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-08T12:39:38.846 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-07-08T12:39:38.846 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-07-08T12:39:38.846 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-07-08T12:39:38.846 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-07-08T12:39:38.847 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-08T12:39:38.847 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:6789,172.21.15.160:6789,172.21.15.160:6790" 2023-07-08T12:39:38.847 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.12:6789,172.21.15.160:6789,172.21.15.160:6790:/ 2023-07-08T12:39:38.847 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=71d270f5-58a7-4223-8d07-37436121729a" will pass to kernel 2023-07-08T12:39:38.848 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-08T12:39:38.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:38.849 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-08T12:39:38.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:38.877 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-08T12:39:38.960 INFO:teuthology.orchestra.run.smithi012.stdout:client5042 2023-07-08T12:39:38.960 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:39:38.961 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-08T12:39:39.017 DEBUG:teuthology.orchestra.run.smithi012:> mktemp -d --suffix=xfstests-dev 2023-07-08T12:39:39.073 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.HvAMMOKiw7xfstests-dev 2023-07-08T12:39:39.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:39.074 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.HvAMMOKiw7xfstests-dev) 2023-07-08T12:39:39.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:39.143 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.HvAMMOKiw7xfstests-dev) 2023-07-08T12:39:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:Cloning into '/tmp/tmp.HvAMMOKiw7xfstests-dev'... 2023-07-08T12:39:39.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:39.865 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2023-07-08T12:39:39.927 DEBUG:teuthology.orchestra.run.smithi012:> mktemp -d --suffix=test 2023-07-08T12:39:39.984 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.RKOPr8TIcotest 2023-07-08T12:39:39.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:39:39.985 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2023-07-08T12:39:40.042 DEBUG:teuthology.orchestra.run.smithi012:> mktemp -d --suffix=scratch 2023-07-08T12:39:40.099 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:39:40.100 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "rhel" and "8.6". 2023-07-08T12:39:40.100 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc indent libtool lvm2 make psmisc quota sed xfsdump xfsprogs libacl-devel libattr-devel libaio-devel libuuid-devel xfsprogs-devel python2 sqlite xfsprogs-qa-devel 2023-07-08T12:39:40.517 INFO:teuthology.orchestra.run.smithi012.stdout:Updating Subscription Management repositories. 2023-07-08T12:39:41.357 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:04:39 ago on Sat 08 Jul 2023 12:35:02 PM UTC. 2023-07-08T12:39:42.780 INFO:teuthology.orchestra.run.smithi012.stdout:Package acl-2.2.53-1.el8.x86_64 is already installed. 2023-07-08T12:39:42.781 INFO:teuthology.orchestra.run.smithi012.stdout:Package attr-2.4.48-3.el8.x86_64 is already installed. 2023-07-08T12:39:42.782 INFO:teuthology.orchestra.run.smithi012.stdout:Package automake-1.16.1-7.el8.noarch is already installed. 2023-07-08T12:39:42.783 INFO:teuthology.orchestra.run.smithi012.stdout:Package bc-1.07.1-5.el8.x86_64 is already installed. 2023-07-08T12:39:42.785 INFO:teuthology.orchestra.run.smithi012.stdout:Package dbench-4.0-20.el8.x86_64 is already installed. 2023-07-08T12:39:42.787 INFO:teuthology.orchestra.run.smithi012.stdout:Package e2fsprogs-1.45.6-4.el8.x86_64 is already installed. 2023-07-08T12:39:42.789 INFO:teuthology.orchestra.run.smithi012.stdout:Package gawk-4.2.1-4.el8.x86_64 is already installed. 2023-07-08T12:39:42.791 INFO:teuthology.orchestra.run.smithi012.stdout:Package gcc-8.5.0-10.1.el8_6.x86_64 is already installed. 2023-07-08T12:39:42.793 INFO:teuthology.orchestra.run.smithi012.stdout:Package libtool-2.4.6-25.el8.x86_64 is already installed. 2023-07-08T12:39:42.794 INFO:teuthology.orchestra.run.smithi012.stdout:Package lvm2-8:2.03.14-3.el8_6.2.x86_64 is already installed. 2023-07-08T12:39:42.796 INFO:teuthology.orchestra.run.smithi012.stdout:Package make-1:4.2.1-11.el8.x86_64 is already installed. 2023-07-08T12:39:42.797 INFO:teuthology.orchestra.run.smithi012.stdout:Package psmisc-23.1-5.el8.x86_64 is already installed. 2023-07-08T12:39:42.798 INFO:teuthology.orchestra.run.smithi012.stdout:Package quota-1:4.04-14.el8.x86_64 is already installed. 2023-07-08T12:39:42.799 INFO:teuthology.orchestra.run.smithi012.stdout:Package sed-4.5-5.el8.x86_64 is already installed. 2023-07-08T12:39:42.801 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-08T12:39:42.802 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-08T12:39:42.803 INFO:teuthology.orchestra.run.smithi012.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-08T12:39:42.804 INFO:teuthology.orchestra.run.smithi012.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-08T12:39:42.805 INFO:teuthology.orchestra.run.smithi012.stdout:Package libaio-devel-0.3.112-1.el8.x86_64 is already installed. 2023-07-08T12:39:42.807 INFO:teuthology.orchestra.run.smithi012.stdout:Package libuuid-devel-2.32.1-35.el8.x86_64 is already installed. 2023-07-08T12:39:42.808 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsprogs-devel-5.0.0-10.el8.x86_64 is already installed. 2023-07-08T12:39:42.810 INFO:teuthology.orchestra.run.smithi012.stdout:Package sqlite-3.26.0-15.el8.x86_64 is already installed. 2023-07-08T12:39:42.815 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-08T12:39:42.938 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2023-07-08T12:39:42.940 INFO:teuthology.orchestra.run.smithi012.stdout:==================================================================================================================== 2023-07-08T12:39:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2023-07-08T12:39:42.941 INFO:teuthology.orchestra.run.smithi012.stdout:==================================================================================================================== 2023-07-08T12:39:42.941 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2023-07-08T12:39:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: dump x86_64 1:0.4-0.36.b46.el8 rhel-8-for-x86_64-baseos-rpms 164 k 2023-07-08T12:39:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: fio x86_64 3.19-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-08T12:39:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: indent x86_64 2.2.13-2.el8 epel 187 k 2023-07-08T12:39:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: python2 x86_64 2.7.18-10.module+el8.6.0+14191+7fdd52cd rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-08T12:39:42.942 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrading: 2023-07-08T12:39:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: sqlite x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 668 k 2023-07-08T12:39:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: sqlite-libs x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 581 k 2023-07-08T12:39:42.942 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2023-07-08T12:39:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: daxctl-libs x86_64 71.1-3.el8 rhel-8-for-x86_64-baseos-rpms 42 k 2023-07-08T12:39:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemblk x86_64 1.6.1-1.el8 rhel-8-for-x86_64-appstream-rpms 92 k 2023-07-08T12:39:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: ndctl-libs x86_64 71.1-3.el8 rhel-8-for-x86_64-baseos-rpms 79 k 2023-07-08T12:39:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: python2-libs x86_64 2.7.18-10.module+el8.6.0+14191+7fdd52cd rhel-8-for-x86_64-appstream-rpms 6.0 M 2023-07-08T12:39:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: python2-pip-wheel noarch 9.0.3-19.module+el8.6.0+13001+ad200bd9 rhel-8-for-x86_64-appstream-rpms 892 k 2023-07-08T12:39:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: python2-setuptools-wheel noarch 39.0.1-13.module+el8.4.0+9442+27d0e81c rhel-8-for-x86_64-appstream-rpms 287 k 2023-07-08T12:39:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: python2-wheel-wheel noarch 1:0.31.1-3.module+el8.5.0+12203+77770ab7 rhel-8-for-x86_64-appstream-rpms 45 k 2023-07-08T12:39:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: rmt x86_64 2:1.5.3-13.el8 rhel-8-for-x86_64-baseos-rpms 51 k 2023-07-08T12:39:42.943 INFO:teuthology.orchestra.run.smithi012.stdout:Installing weak dependencies: 2023-07-08T12:39:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: python2-pip noarch 9.0.3-19.module+el8.6.0+13001+ad200bd9 rhel-8-for-x86_64-appstream-rpms 1.6 M 2023-07-08T12:39:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: python2-setuptools noarch 39.0.1-13.module+el8.4.0+9442+27d0e81c rhel-8-for-x86_64-appstream-rpms 642 k 2023-07-08T12:39:42.944 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling module streams: 2023-07-08T12:39:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: python27 2.7 2023-07-08T12:39:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:42.944 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2023-07-08T12:39:42.944 INFO:teuthology.orchestra.run.smithi012.stdout:==================================================================================================================== 2023-07-08T12:39:42.945 INFO:teuthology.orchestra.run.smithi012.stdout:Install 14 Packages 2023-07-08T12:39:42.945 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrade 2 Packages 2023-07-08T12:39:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:42.945 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 12 M 2023-07-08T12:39:42.946 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2023-07-08T12:39:43.108 INFO:teuthology.orchestra.run.smithi012.stdout:(1/16): rmt-1.5.3-13.el8.x86_64.rpm 553 kB/s | 51 kB 00:00 2023-07-08T12:39:43.171 INFO:teuthology.orchestra.run.smithi012.stdout:(2/16): dump-0.4-0.36.b46.el8.x86_64.rpm 1.0 MB/s | 164 kB 00:00 2023-07-08T12:39:43.200 INFO:teuthology.orchestra.run.smithi012.stdout:(3/16): daxctl-libs-71.1-3.el8.x86_64.rpm 477 kB/s | 42 kB 00:00 2023-07-08T12:39:43.218 INFO:teuthology.orchestra.run.smithi012.stdout:(4/16): ndctl-libs-71.1-3.el8.x86_64.rpm 1.7 MB/s | 79 kB 00:00 2023-07-08T12:39:43.259 INFO:teuthology.orchestra.run.smithi012.stdout:(5/16): python2-setuptools-wheel-39.0.1-13.modu 4.8 MB/s | 287 kB 00:00 2023-07-08T12:39:43.293 INFO:teuthology.orchestra.run.smithi012.stdout:(6/16): python2-setuptools-39.0.1-13.module+el8 8.3 MB/s | 642 kB 00:00 2023-07-08T12:39:43.319 INFO:teuthology.orchestra.run.smithi012.stdout:(7/16): python2-wheel-wheel-0.31.1-3.module+el8 760 kB/s | 45 kB 00:00 2023-07-08T12:39:43.352 INFO:teuthology.orchestra.run.smithi012.stdout:(8/16): fio-3.19-3.el8.x86_64.rpm 9.0 MB/s | 538 kB 00:00 2023-07-08T12:39:43.377 INFO:teuthology.orchestra.run.smithi012.stdout:(9/16): libpmemblk-1.6.1-1.el8.x86_64.rpm 1.5 MB/s | 92 kB 00:00 2023-07-08T12:39:43.403 INFO:teuthology.orchestra.run.smithi012.stdout:(10/16): python2-2.7.18-10.module+el8.6.0+14191 2.1 MB/s | 110 kB 00:00 2023-07-08T12:39:43.495 INFO:teuthology.orchestra.run.smithi012.stdout:(11/16): python2-pip-9.0.3-19.module+el8.6.0+13 18 MB/s | 1.6 MB 00:00 2023-07-08T12:39:43.620 INFO:teuthology.orchestra.run.smithi012.stdout:(12/16): python2-libs-2.7.18-10.module+el8.6.0+ 25 MB/s | 6.0 MB 00:00 2023-07-08T12:39:43.645 INFO:teuthology.orchestra.run.smithi012.stdout:(13/16): indent-2.2.13-2.el8.x86_64.rpm 296 kB/s | 187 kB 00:00 2023-07-08T12:39:43.687 INFO:teuthology.orchestra.run.smithi012.stdout:(14/16): python2-pip-wheel-9.0.3-19.module+el8. 4.5 MB/s | 892 kB 00:00 2023-07-08T12:39:43.729 INFO:teuthology.orchestra.run.smithi012.stdout:(15/16): sqlite-3.26.0-16.el8_6.x86_64.rpm 6.0 MB/s | 668 kB 00:00 2023-07-08T12:39:43.763 INFO:teuthology.orchestra.run.smithi012.stdout:(16/16): sqlite-libs-3.26.0-16.el8_6.x86_64.rpm 4.8 MB/s | 581 kB 00:00 2023-07-08T12:39:43.763 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2023-07-08T12:39:43.764 INFO:teuthology.orchestra.run.smithi012.stdout:Total 14 MB/s | 12 MB 00:00 2023-07-08T12:39:43.973 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2023-07-08T12:39:44.016 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2023-07-08T12:39:44.016 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2023-07-08T12:39:44.213 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2023-07-08T12:39:44.218 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2023-07-08T12:39:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2023-07-08T12:39:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : sqlite-libs-3.26.0-16.el8_6.x86_64 1/18 2023-07-08T12:39:45.355 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : daxctl-libs-71.1-3.el8.x86_64 2/18 2023-07-08T12:39:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: daxctl-libs-71.1-3.el8.x86_64 2/18 2023-07-08T12:39:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ndctl-libs-71.1-3.el8.x86_64 3/18 2023-07-08T12:39:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ndctl-libs-71.1-3.el8.x86_64 3/18 2023-07-08T12:39:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libpmemblk-1.6.1-1.el8.x86_64 4/18 2023-07-08T12:39:46.001 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libpmemblk-1.6.1-1.el8.x86_64 4/18 2023-07-08T12:39:46.142 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python2-pip-wheel-9.0.3-19.module+el8.6.0+13001+ad 5/18 2023-07-08T12:39:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python2-setuptools-wheel-39.0.1-13.module+el8.4.0+ 6/18 2023-07-08T12:39:47.149 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python2-setuptools-39.0.1-13.module+el8.4.0+9442+2 7/18 2023-07-08T12:39:47.432 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python2-pip-9.0.3-19.module+el8.6.0+13001+ad200bd9 8/18 2023-07-08T12:39:47.890 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x8 9/18 2023-07-08T12:39:49.087 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x8 9/18 2023-07-08T12:39:49.402 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python2-libs-2.7.18-10.module+el8.6.0+14191+7fdd52 10/18 2023-07-08T12:39:49.589 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rmt-2:1.5.3-13.el8.x86_64 11/18 2023-07-08T12:39:49.919 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : dump-1:0.4-0.36.b46.el8.x86_64 12/18 2023-07-08T12:39:50.212 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : fio-3.19-3.el8.x86_64 13/18 2023-07-08T12:39:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : sqlite-3.26.0-16.el8_6.x86_64 14/18 2023-07-08T12:39:50.627 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python2-wheel-wheel-1:0.31.1-3.module+el8.5.0+1220 15/18 2023-07-08T12:39:50.804 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : indent-2.2.13-2.el8.x86_64 16/18 2023-07-08T12:39:50.962 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : sqlite-3.26.0-15.el8.x86_64 17/18 2023-07-08T12:39:51.102 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : sqlite-libs-3.26.0-15.el8.x86_64 18/18 2023-07-08T12:39:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: sqlite-libs-3.26.0-15.el8.x86_64 18/18 2023-07-08T12:39:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : indent-2.2.13-2.el8.x86_64 1/18 2023-07-08T12:39:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : dump-1:0.4-0.36.b46.el8.x86_64 2/18 2023-07-08T12:39:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rmt-2:1.5.3-13.el8.x86_64 3/18 2023-07-08T12:39:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : daxctl-libs-71.1-3.el8.x86_64 4/18 2023-07-08T12:39:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ndctl-libs-71.1-3.el8.x86_64 5/18 2023-07-08T12:39:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python2-setuptools-wheel-39.0.1-13.module+el8.4.0+ 6/18 2023-07-08T12:39:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python2-setuptools-39.0.1-13.module+el8.4.0+9442+2 7/18 2023-07-08T12:39:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python2-wheel-wheel-1:0.31.1-3.module+el8.5.0+1220 8/18 2023-07-08T12:39:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : fio-3.19-3.el8.x86_64 9/18 2023-07-08T12:39:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libpmemblk-1.6.1-1.el8.x86_64 10/18 2023-07-08T12:39:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x8 11/18 2023-07-08T12:39:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python2-libs-2.7.18-10.module+el8.6.0+14191+7fdd52 12/18 2023-07-08T12:39:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python2-pip-9.0.3-19.module+el8.6.0+13001+ad200bd9 13/18 2023-07-08T12:39:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python2-pip-wheel-9.0.3-19.module+el8.6.0+13001+ad 14/18 2023-07-08T12:39:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : sqlite-3.26.0-16.el8_6.x86_64 15/18 2023-07-08T12:39:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : sqlite-3.26.0-15.el8.x86_64 16/18 2023-07-08T12:39:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : sqlite-libs-3.26.0-16.el8_6.x86_64 17/18 2023-07-08T12:39:52.708 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : sqlite-libs-3.26.0-15.el8.x86_64 18/18 2023-07-08T12:39:52.709 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:04:50 ago on Sat 08 Jul 2023 12:35:02 PM UTC. 2023-07-08T12:39:55.418 INFO:teuthology.orchestra.run.smithi012.stdout:Installed products updated. 2023-07-08T12:39:55.627 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:55.627 INFO:teuthology.orchestra.run.smithi012.stdout:Upgraded: 2023-07-08T12:39:55.628 INFO:teuthology.orchestra.run.smithi012.stdout: sqlite-3.26.0-16.el8_6.x86_64 sqlite-libs-3.26.0-16.el8_6.x86_64 2023-07-08T12:39:55.628 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2023-07-08T12:39:55.628 INFO:teuthology.orchestra.run.smithi012.stdout: daxctl-libs-71.1-3.el8.x86_64 2023-07-08T12:39:55.628 INFO:teuthology.orchestra.run.smithi012.stdout: dump-1:0.4-0.36.b46.el8.x86_64 2023-07-08T12:39:55.628 INFO:teuthology.orchestra.run.smithi012.stdout: fio-3.19-3.el8.x86_64 2023-07-08T12:39:55.628 INFO:teuthology.orchestra.run.smithi012.stdout: indent-2.2.13-2.el8.x86_64 2023-07-08T12:39:55.629 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemblk-1.6.1-1.el8.x86_64 2023-07-08T12:39:55.629 INFO:teuthology.orchestra.run.smithi012.stdout: ndctl-libs-71.1-3.el8.x86_64 2023-07-08T12:39:55.629 INFO:teuthology.orchestra.run.smithi012.stdout: python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x86_64 2023-07-08T12:39:55.629 INFO:teuthology.orchestra.run.smithi012.stdout: python2-libs-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x86_64 2023-07-08T12:39:55.629 INFO:teuthology.orchestra.run.smithi012.stdout: python2-pip-9.0.3-19.module+el8.6.0+13001+ad200bd9.noarch 2023-07-08T12:39:55.629 INFO:teuthology.orchestra.run.smithi012.stdout: python2-pip-wheel-9.0.3-19.module+el8.6.0+13001+ad200bd9.noarch 2023-07-08T12:39:55.629 INFO:teuthology.orchestra.run.smithi012.stdout: python2-setuptools-39.0.1-13.module+el8.4.0+9442+27d0e81c.noarch 2023-07-08T12:39:55.630 INFO:teuthology.orchestra.run.smithi012.stdout: python2-setuptools-wheel-39.0.1-13.module+el8.4.0+9442+27d0e81c.noarch 2023-07-08T12:39:55.630 INFO:teuthology.orchestra.run.smithi012.stdout: python2-wheel-wheel-1:0.31.1-3.module+el8.5.0+12203+77770ab7.noarch 2023-07-08T12:39:55.630 INFO:teuthology.orchestra.run.smithi012.stdout: rmt-2:1.5.3-13.el8.x86_64 2023-07-08T12:39:55.630 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:39:55.630 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2023-07-08T12:39:55.810 DEBUG:teuthology.orchestra.run.smithi012:> sudo useradd -m fsgqa 2023-07-08T12:39:55.841 INFO:teuthology.orchestra.run.smithi012.stderr:useradd: user 'fsgqa' already exists 2023-07-08T12:39:55.842 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-07-08T12:39:55.843 DEBUG:teuthology.orchestra.run.smithi012:> sudo groupadd fsgqa 2023-07-08T12:39:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:groupadd: group 'fsgqa' already exists 2023-07-08T12:39:55.910 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-07-08T12:39:55.911 DEBUG:teuthology.orchestra.run.smithi012:> sudo useradd fsgqa2 2023-07-08T12:39:56.989 DEBUG:teuthology.orchestra.run.smithi012:> sudo useradd 123456-fsgqa 2023-07-08T12:39:59.730 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2023-07-08T12:39:59.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.865+0000 7f652fded700 1 Processor -- start 2023-07-08T12:39:59.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.865+0000 7f652fded700 1 -- start start 2023-07-08T12:39:59.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.866+0000 7f652fded700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f65280cbb60 0x7f65280ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:59.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.866+0000 7f652fded700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652806cd50 con 0x7f652806a640 2023-07-08T12:39:59.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.866+0000 7f652fded700 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65280c8ff0 con 0x7f65281588b0 2023-07-08T12:39:59.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.866+0000 7f652fded700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652806d280 con 0x7f65280cbb60 2023-07-08T12:39:59.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.866+0000 7f652d388700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f652806a640 0x7f65280c9d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39360/0 (socket says 172.21.15.12:39360) 2023-07-08T12:39:59.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.866+0000 7f652d388700 1 -- 172.21.15.12:0/244625185 learned_addr learned my addr 172.21.15.12:0/244625185 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:39:59.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.866+0000 7f652e38a700 1 --2- 172.21.15.12:0/244625185 >> v2:172.21.15.160:3300/0 conn(0x7f65280cbb60 0x7f65280ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:59.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.866+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898226248 0 0) 0x7f652806cd50 con 0x7f652806a640 2023-07-08T12:39:59.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.866+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65140035b0 con 0x7f652806a640 2023-07-08T12:39:59.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1049901523 0 0) 0x7f65140035b0 con 0x7f652806a640 2023-07-08T12:39:59.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 >> v2:172.21.15.160:3300/0 conn(0x7f65280cbb60 msgr2=0x7f65280ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:59.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 --2- 172.21.15.12:0/244625185 >> v2:172.21.15.160:3300/0 conn(0x7f65280cbb60 0x7f65280ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:59.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 >> v1:172.21.15.160:6789/0 conn(0x7f65281588b0 legacy=0x7f6528069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:59.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652806d880 con 0x7f652806a640 2023-07-08T12:39:59.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f6524001670 con 0x7f652806a640 2023-07-08T12:39:59.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 >> v1:172.21.15.12:6789/0 conn(0x7f652806a640 legacy=0x7f65280c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:59.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 --2- 172.21.15.12:0/244625185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6514003a00 0x7f6514003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:59.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 --2- 172.21.15.12:0/244625185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6514004850 0x7f6514004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:59.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 --2- 172.21.15.12:0/244625185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6514005200 0x7f651400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:59.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652806d280 con 0x7f6514004850 2023-07-08T12:39:59.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651400c110 con 0x7f6514005200 2023-07-08T12:39:59.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651400c2d0 con 0x7f6514003a00 2023-07-08T12:39:59.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652d388700 1 --2- 172.21.15.12:0/244625185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6514004850 0x7f6514004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:59.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652db89700 1 --2- 172.21.15.12:0/244625185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6514003a00 0x7f6514003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:59.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652e38a700 1 --2- 172.21.15.12:0/244625185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6514005200 0x7f651400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:59.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652d388700 1 -- 172.21.15.12:0/244625185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6514003a00 msgr2=0x7f6514003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:59.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652d388700 1 --2- 172.21.15.12:0/244625185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6514003a00 0x7f6514003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:59.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652d388700 1 -- 172.21.15.12:0/244625185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6514005200 msgr2=0x7f651400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:39:59.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652d388700 1 --2- 172.21.15.12:0/244625185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6514005200 0x7f651400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:59.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652e38a700 1 -- 172.21.15.12:0/244625185 reap_dead start 2023-07-08T12:39:59.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652d388700 1 -- 172.21.15.12:0/244625185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65140035b0 con 0x7f6514004850 2023-07-08T12:39:59.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652d388700 1 --2- 172.21.15.12:0/244625185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6514004850 0x7f6514004cc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6528070110 tx=0x7f6514005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1e456734461f04 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:59.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.867+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524007c10 con 0x7f6514004850 2023-07-08T12:39:59.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.868+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652400cce0 con 0x7f6514004850 2023-07-08T12:39:59.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.868+0000 7f652cb87700 1 -- 172.21.15.12:0/244625185 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524005ba0 con 0x7f6514004850 2023-07-08T12:39:59.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.868+0000 7f652fded700 1 -- 172.21.15.12:0/244625185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6514004850 msgr2=0x7f6514004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:59.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.868+0000 7f652fded700 1 --2- 172.21.15.12:0/244625185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6514004850 0x7f6514004cc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6528070110 tx=0x7f6514005880 comp rx=0 tx=0).stop 2023-07-08T12:39:59.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.868+0000 7f652fded700 1 -- 172.21.15.12:0/244625185 shutdown_connections 2023-07-08T12:39:59.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.868+0000 7f652fded700 1 --2- 172.21.15.12:0/244625185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6514004850 0x7f6514004cc0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:59.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.868+0000 7f652fded700 1 -- 172.21.15.12:0/244625185 >> 172.21.15.12:0/244625185 conn(0x7f65280b0390 msgr2=0x7f65280b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:39:59.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.868+0000 7f652fded700 1 -- 172.21.15.12:0/244625185 shutdown_connections 2023-07-08T12:39:59.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.868+0000 7f652fded700 1 -- 172.21.15.12:0/244625185 wait complete. 2023-07-08T12:39:59.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.869+0000 7f652fded700 1 Processor -- start 2023-07-08T12:39:59.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.869+0000 7f652fded700 1 -- start start 2023-07-08T12:39:59.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.870+0000 7f652fded700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6514004850 0x7f652815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:59.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.870+0000 7f652fded700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65281f1230 0x7f65281f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:59.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.870+0000 7f652fded700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65280c2e20 0x7f65280c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:59.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.870+0000 7f652fded700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652815afa0 con 0x7f6514004850 2023-07-08T12:39:59.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.870+0000 7f652fded700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65281f1be0 con 0x7f65280c2e20 2023-07-08T12:39:59.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.870+0000 7f652fded700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652815b900 con 0x7f65281f1230 2023-07-08T12:39:59.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652db89700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6514004850 0x7f652815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:59.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652db89700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6514004850 0x7f652815b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51974/0 (socket says 172.21.15.12:51974) 2023-07-08T12:39:59.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652db89700 1 -- 172.21.15.12:0/419430784 learned_addr learned my addr 172.21.15.12:0/419430784 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:39:59.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652e38a700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65280c2e20 0x7f65280c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:59.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652d388700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65281f1230 0x7f65281f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:59.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652db89700 1 -- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65281f1230 msgr2=0x7f65281f16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:59.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652db89700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65281f1230 0x7f65281f16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652db89700 1 -- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65280c2e20 msgr2=0x7f65280c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:39:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652db89700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65280c2e20 0x7f65280c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:39:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652db89700 1 -- 172.21.15.12:0/419430784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652815a360 con 0x7f6514004850 2023-07-08T12:39:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652e38a700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65280c2e20 0x7f65280c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-08T12:39:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652db89700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6514004850 0x7f652815b3c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f651800f850 tx=0x7f651800fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f651e7fc700 1 -- 172.21.15.12:0/419430784 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65180050d0 con 0x7f6514004850 2023-07-08T12:39:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f651e7fc700 1 -- 172.21.15.12:0/419430784 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6518005230 con 0x7f6514004850 2023-07-08T12:39:59.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652fded700 1 -- 172.21.15.12:0/419430784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652815a600 con 0x7f6514004850 2023-07-08T12:39:59.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f651e7fc700 1 -- 172.21.15.12:0/419430784 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518003680 con 0x7f6514004850 2023-07-08T12:39:59.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.871+0000 7f652fded700 1 -- 172.21.15.12:0/419430784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652815abc0 con 0x7f6514004850 2023-07-08T12:39:59.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.872+0000 7f652fded700 1 -- 172.21.15.12:0/419430784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652815b210 con 0x7f6514004850 2023-07-08T12:39:59.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.873+0000 7f651e7fc700 1 -- 172.21.15.12:0/419430784 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f65180037e0 con 0x7f6514004850 2023-07-08T12:39:59.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.873+0000 7f651e7fc700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f650c06ace0 0x7f650c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:39:59.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.873+0000 7f651e7fc700 1 -- 172.21.15.12:0/419430784 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f6518089300 con 0x7f6514004850 2023-07-08T12:39:59.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.873+0000 7f652d388700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f650c06ace0 0x7f650c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:39:59.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.874+0000 7f652d388700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f650c06ace0 0x7f650c06d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f65280b3c50 tx=0x7f6524007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:39:59.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:39:59.878+0000 7f651e7fc700 1 -- 172.21.15.12:0/419430784 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f65180542c0 con 0x7f6514004850 2023-07-08T12:40:00.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.158+0000 7f652fded700 1 -- 172.21.15.12:0/419430784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f652815ad80 con 0x7f6514004850 2023-07-08T12:40:00.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.158+0000 7f651e7fc700 1 -- 172.21.15.12:0/419430784 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+1276 (secure 0 0 0) 0x7f65180578e0 con 0x7f6514004850 2023-07-08T12:40:00.161 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:40:00.162 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":1,"fsid":"71d270f5-58a7-4223-8d07-37436121729a","modified":"2023-07-08T12:37:07.815136Z","created":"2023-07-08T12:37:07.815136Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-07-08T12:40:00.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.160+0000 7f652fded700 1 -- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f650c06ace0 msgr2=0x7f650c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:40:00.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.160+0000 7f652fded700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f650c06ace0 0x7f650c06d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f65280b3c50 tx=0x7f6524007000 comp rx=0 tx=0).stop 2023-07-08T12:40:00.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.160+0000 7f652fded700 1 -- 172.21.15.12:0/419430784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6514004850 msgr2=0x7f652815b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:40:00.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.160+0000 7f652fded700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6514004850 0x7f652815b3c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f651800f850 tx=0x7f651800fc10 comp rx=0 tx=0).stop 2023-07-08T12:40:00.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.160+0000 7f652fded700 1 -- 172.21.15.12:0/419430784 shutdown_connections 2023-07-08T12:40:00.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.160+0000 7f652fded700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f650c06ace0 0x7f650c06d190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.160+0000 7f652fded700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65280c2e20 0x7f65280c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.160+0000 7f652fded700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6514004850 0x7f652815b3c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.160+0000 7f652fded700 1 --2- 172.21.15.12:0/419430784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65281f1230 0x7f65281f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.160+0000 7f652fded700 1 -- 172.21.15.12:0/419430784 >> 172.21.15.12:0/419430784 conn(0x7f65280b0390 msgr2=0x7f65280ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:40:00.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.160+0000 7f652fded700 1 -- 172.21.15.12:0/419430784 shutdown_connections 2023-07-08T12:40:00.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.160+0000 7f652fded700 1 -- 172.21.15.12:0/419430784 wait complete. 2023-07-08T12:40:00.167 INFO:teuthology.orchestra.run.smithi012.stderr:dumped monmap epoch 1 2023-07-08T12:40:00.175 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.admin 2023-07-08T12:40:00.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.301+0000 7f98e5fd7700 1 Processor -- start 2023-07-08T12:40:00.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.301+0000 7f98e5fd7700 1 -- start start 2023-07-08T12:40:00.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.302+0000 7f98e5fd7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e0160790 0x7f98e00cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:40:00.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.302+0000 7f98e5fd7700 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f98e0069f30 0x7f98e006a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:40:00.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.302+0000 7f98e5fd7700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e015c890 con 0x7f98e0159c30 2023-07-08T12:40:00.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.302+0000 7f98e5fd7700 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e015c730 con 0x7f98e0069f30 2023-07-08T12:40:00.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.302+0000 7f98e5fd7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00cf590 con 0x7f98e0160790 2023-07-08T12:40:00.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.302+0000 7f98df7fe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f98e0159c30 0x7f98e006ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39378/0 (socket says 172.21.15.12:39378) 2023-07-08T12:40:00.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.302+0000 7f98df7fe700 1 -- 172.21.15.12:0/2455751913 learned_addr learned my addr 172.21.15.12:0/2455751913 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-07-08T12:40:00.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.302+0000 7f98dffff700 1 --2- 172.21.15.12:0/2455751913 >> v2:172.21.15.160:3300/0 conn(0x7f98e0069f30 0x7f98e006a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:40:00.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.302+0000 7f98deffd700 1 --2- 172.21.15.12:0/2455751913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e0160790 0x7f98e00cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:40:00.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.302+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359198223 0 0) 0x7f98e015c890 con 0x7f98e0159c30 2023-07-08T12:40:00.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.302+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98cc0035b0 con 0x7f98e0159c30 2023-07-08T12:40:00.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3574397113 0 0) 0x7f98cc0035b0 con 0x7f98e0159c30 2023-07-08T12:40:00.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e0160790 msgr2=0x7f98e00cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:40:00.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 --2- 172.21.15.12:0/2455751913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e0160790 0x7f98e00cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 >> v2:172.21.15.160:3300/0 conn(0x7f98e0069f30 msgr2=0x7f98e006a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:40:00.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 --2- 172.21.15.12:0/2455751913 >> v2:172.21.15.160:3300/0 conn(0x7f98e0069f30 0x7f98e006a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e006f620 con 0x7f98e0159c30 2023-07-08T12:40:00.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3656454722 0 0) 0x7f98d0001670 con 0x7f98e0159c30 2023-07-08T12:40:00.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98dffff700 1 --2- 172.21.15.12:0/2455751913 >> v2:172.21.15.160:3300/0 conn(0x7f98e0069f30 0x7f98e006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-08T12:40:00.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 >> v1:172.21.15.12:6789/0 conn(0x7f98e0159c30 legacy=0x7f98e006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:40:00.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 --2- 172.21.15.12:0/2455751913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e0160790 0x7f98cc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:40:00.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 --2- 172.21.15.12:0/2455751913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc003fb0 0x7f98cc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:40:00.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 --2- 172.21.15.12:0/2455751913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc004d90 0x7f98cc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:40:00.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e015c730 con 0x7f98cc003fb0 2023-07-08T12:40:00.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00cf590 con 0x7f98cc004d90 2023-07-08T12:40:00.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc00ba10 con 0x7f98e0160790 2023-07-08T12:40:00.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98deffd700 1 --2- 172.21.15.12:0/2455751913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc003fb0 0x7f98cc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:40:00.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98df7fe700 1 --2- 172.21.15.12:0/2455751913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e0160790 0x7f98cc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:40:00.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98dffff700 1 --2- 172.21.15.12:0/2455751913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc004d90 0x7f98cc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:40:00.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98deffd700 1 -- 172.21.15.12:0/2455751913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e0160790 msgr2=0x7f98cc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:40:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98deffd700 1 --2- 172.21.15.12:0/2455751913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e0160790 0x7f98cc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98deffd700 1 -- 172.21.15.12:0/2455751913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc004d90 msgr2=0x7f98cc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-08T12:40:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98deffd700 1 --2- 172.21.15.12:0/2455751913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc004d90 0x7f98cc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98deffd700 1 -- 172.21.15.12:0/2455751913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc0035b0 con 0x7f98cc003fb0 2023-07-08T12:40:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.303+0000 7f98deffd700 1 --2- 172.21.15.12:0/2455751913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc003fb0 0x7f98cc004850 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009b30 tx=0x7f98cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f14579ee4c2d7c server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:40:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.304+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c800e6c0 con 0x7f98cc003fb0 2023-07-08T12:40:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.304+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c800e820 con 0x7f98cc003fb0 2023-07-08T12:40:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.304+0000 7f98de7fc700 1 -- 172.21.15.12:0/2455751913 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c800eb90 con 0x7f98cc003fb0 2023-07-08T12:40:00.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.304+0000 7f98e5fd7700 1 -- 172.21.15.12:0/2455751913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc003fb0 msgr2=0x7f98cc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:40:00.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.304+0000 7f98e5fd7700 1 --2- 172.21.15.12:0/2455751913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98cc003fb0 0x7f98cc004850 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009b30 tx=0x7f98cc005880 comp rx=0 tx=0).stop 2023-07-08T12:40:00.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.304+0000 7f98dffff700 1 -- 172.21.15.12:0/2455751913 reap_dead start 2023-07-08T12:40:00.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.304+0000 7f98e5fd7700 1 -- 172.21.15.12:0/2455751913 shutdown_connections 2023-07-08T12:40:00.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.304+0000 7f98e5fd7700 1 -- 172.21.15.12:0/2455751913 >> 172.21.15.12:0/2455751913 conn(0x7f98e00cc290 msgr2=0x7f98e00cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:40:00.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.304+0000 7f98e5fd7700 1 -- 172.21.15.12:0/2455751913 shutdown_connections 2023-07-08T12:40:00.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.304+0000 7f98e5fd7700 1 -- 172.21.15.12:0/2455751913 wait complete. 2023-07-08T12:40:00.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.306+0000 7f98e5fd7700 1 Processor -- start 2023-07-08T12:40:00.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.306+0000 7f98e5fd7700 1 -- start start 2023-07-08T12:40:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98e5fd7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc003fb0 0x7f98e00c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:40:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98e5fd7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00c4860 0x7f98e0150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:40:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98e5fd7700 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e00b38e0 0x7f98e015e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:40:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98e5fd7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00b4ff0 con 0x7f98e00c4860 2023-07-08T12:40:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98e5fd7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00c5900 con 0x7f98cc003fb0 2023-07-08T12:40:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98e5fd7700 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00c4cf0 con 0x7f98e00b38e0 2023-07-08T12:40:00.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98deffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00c4860 0x7f98e0150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:40:00.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98deffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00c4860 0x7f98e0150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51996/0 (socket says 172.21.15.12:51996) 2023-07-08T12:40:00.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98deffd700 1 -- 172.21.15.12:0/1847604783 learned_addr learned my addr 172.21.15.12:0/1847604783 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-07-08T12:40:00.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98df7fe700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc003fb0 0x7f98e00c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:40:00.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98dffff700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e00b38e0 0x7f98e015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:40:00.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98deffd700 1 -- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e00b38e0 msgr2=0x7f98e015e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:40:00.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98deffd700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e00b38e0 0x7f98e015e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98deffd700 1 -- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc003fb0 msgr2=0x7f98e00c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:40:00.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98deffd700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc003fb0 0x7f98e00c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98deffd700 1 -- 172.21.15.12:0/1847604783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e00c4fd0 con 0x7f98e00c4860 2023-07-08T12:40:00.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98df7fe700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc003fb0 0x7f98e00c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:40:00.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.307+0000 7f98dffff700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e00b38e0 0x7f98e015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-08T12:40:00.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.308+0000 7f98deffd700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00c4860 0x7f98e0150cc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f98c800e4b0 tx=0x7f98c80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:40:00.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.308+0000 7f98d7fff700 1 -- 172.21.15.12:0/1847604783 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8021070 con 0x7f98e00c4860 2023-07-08T12:40:00.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.308+0000 7f98d7fff700 1 -- 172.21.15.12:0/1847604783 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c8004540 con 0x7f98e00c4860 2023-07-08T12:40:00.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.308+0000 7f98e5fd7700 1 -- 172.21.15.12:0/1847604783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e00c52b0 con 0x7f98e00c4860 2023-07-08T12:40:00.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.308+0000 7f98d7fff700 1 -- 172.21.15.12:0/1847604783 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c800e6c0 con 0x7f98e00c4860 2023-07-08T12:40:00.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.308+0000 7f98e5fd7700 1 -- 172.21.15.12:0/1847604783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e0151200 con 0x7f98e00c4860 2023-07-08T12:40:00.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.309+0000 7f98e5fd7700 1 -- 172.21.15.12:0/1847604783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e00c5a90 con 0x7f98e00c4860 2023-07-08T12:40:00.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.309+0000 7f98d7fff700 1 -- 172.21.15.12:0/1847604783 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f98c8025070 con 0x7f98e00c4860 2023-07-08T12:40:00.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.309+0000 7f98d7fff700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f98c006ace0 0x7f98c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-08T12:40:00.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.310+0000 7f98d7fff700 1 -- 172.21.15.12:0/1847604783 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f98c808a740 con 0x7f98e00c4860 2023-07-08T12:40:00.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.310+0000 7f98df7fe700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f98c006ace0 0x7f98c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-08T12:40:00.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.310+0000 7f98df7fe700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f98c006ace0 0x7f98c006d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f98d0000c00 tx=0x7f98d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-08T12:40:00.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.314+0000 7f98d7fff700 1 -- 172.21.15.12:0/1847604783 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f98c8055820 con 0x7f98e00c4860 2023-07-08T12:40:00.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.593+0000 7f98e5fd7700 1 -- 172.21.15.12:0/1847604783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.admin"} v 0) v1 -- 0x7f98e00cd450 con 0x7f98e00c4860 2023-07-08T12:40:00.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.593+0000 7f98d7fff700 1 -- 172.21.15.12:0/1847604783 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.admin"}]=0 v11) v1 ==== 92+0+63 (secure 0 0 0) 0x7f98c8058e40 con 0x7f98e00c4860 2023-07-08T12:40:00.596 INFO:teuthology.orchestra.run.smithi012.stdout:[client.admin] 2023-07-08T12:40:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== 2023-07-08T12:40:00.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.595+0000 7f98e5fd7700 1 -- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f98c006ace0 msgr2=0x7f98c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:40:00.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.595+0000 7f98e5fd7700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f98c006ace0 0x7f98c006d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f98d0000c00 tx=0x7f98d0009040 comp rx=0 tx=0).stop 2023-07-08T12:40:00.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.595+0000 7f98e5fd7700 1 -- 172.21.15.12:0/1847604783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00c4860 msgr2=0x7f98e0150cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-08T12:40:00.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.595+0000 7f98e5fd7700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00c4860 0x7f98e0150cc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f98c800e4b0 tx=0x7f98c80047c0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.595+0000 7f98e5fd7700 1 -- 172.21.15.12:0/1847604783 shutdown_connections 2023-07-08T12:40:00.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.595+0000 7f98e5fd7700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:6832/550875184,v1:172.21.15.160:6833/550875184] conn(0x7f98c006ace0 0x7f98c006d190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.595+0000 7f98e5fd7700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e00b38e0 0x7f98e015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.595+0000 7f98e5fd7700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc003fb0 0x7f98e00c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.595+0000 7f98e5fd7700 1 --2- 172.21.15.12:0/1847604783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00c4860 0x7f98e0150cc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-08T12:40:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.595+0000 7f98e5fd7700 1 -- 172.21.15.12:0/1847604783 >> 172.21.15.12:0/1847604783 conn(0x7f98e00cc290 msgr2=0x7f98e0071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-08T12:40:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.596+0000 7f98e5fd7700 1 -- 172.21.15.12:0/1847604783 shutdown_connections 2023-07-08T12:40:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-07-08T12:40:00.596+0000 7f98e5fd7700 1 -- 172.21.15.12:0/1847604783 wait complete. 2023-07-08T12:40:00.610 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:40:00.611 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/tmp/tmp.HvAMMOKiw7xfstests-dev/local.config 2023-07-08T12:40:00.639 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph export TEST_DEV=172.21.15.12:6789:/test export TEST_DIR=/tmp/tmp.RKOPr8TIcotest export SCRATCH_DEV=172.21.15.12:6789:/scratch export SCRATCH_MNT=/tmp/tmp.0MVtsqxQ3pscratch export CEPHFS_MOUNT_OPTIONS="-o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g==" 2023-07-08T12:40:00.639 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-07-08T12:40:00.639 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/tmp/tmp.HvAMMOKiw7xfstests-dev/ceph.exclude 2023-07-08T12:40:00.706 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo make) 2023-07-08T12:40:00.788 INFO:teuthology.orchestra.run.smithi012.stdout:libtoolize -cfi 2023-07-08T12:40:01.158 INFO:teuthology.orchestra.run.smithi012.stdout:libtoolize: putting auxiliary files in '.'. 2023-07-08T12:40:01.159 INFO:teuthology.orchestra.run.smithi012.stdout:libtoolize: copying file './config.guess' 2023-07-08T12:40:01.162 INFO:teuthology.orchestra.run.smithi012.stdout:libtoolize: copying file './config.sub' 2023-07-08T12:40:01.165 INFO:teuthology.orchestra.run.smithi012.stdout:libtoolize: copying file './install-sh' 2023-07-08T12:40:01.169 INFO:teuthology.orchestra.run.smithi012.stdout:libtoolize: copying file './ltmain.sh' 2023-07-08T12:40:01.190 INFO:teuthology.orchestra.run.smithi012.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2023-07-08T12:40:01.190 INFO:teuthology.orchestra.run.smithi012.stdout:libtoolize: copying file 'm4/libtool.m4' 2023-07-08T12:40:01.194 INFO:teuthology.orchestra.run.smithi012.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2023-07-08T12:40:01.199 INFO:teuthology.orchestra.run.smithi012.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2023-07-08T12:40:01.203 INFO:teuthology.orchestra.run.smithi012.stdout:libtoolize: copying file 'm4/ltversion.m4' 2023-07-08T12:40:01.208 INFO:teuthology.orchestra.run.smithi012.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2023-07-08T12:40:01.214 INFO:teuthology.orchestra.run.smithi012.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2023-07-08T12:40:01.215 INFO:teuthology.orchestra.run.smithi012.stdout:cp include/install-sh . 2023-07-08T12:40:01.216 INFO:teuthology.orchestra.run.smithi012.stdout:aclocal -I m4 2023-07-08T12:40:02.440 INFO:teuthology.orchestra.run.smithi012.stdout:autoheader 2023-07-08T12:40:02.764 INFO:teuthology.orchestra.run.smithi012.stdout:autoconf 2023-07-08T12:40:02.929 INFO:teuthology.orchestra.run.smithi012.stdout:./configure \ 2023-07-08T12:40:02.930 INFO:teuthology.orchestra.run.smithi012.stdout: --libexecdir=/usr/lib \ 2023-07-08T12:40:02.930 INFO:teuthology.orchestra.run.smithi012.stdout: --exec_prefix=/var/lib 2023-07-08T12:40:03.380 INFO:teuthology.orchestra.run.smithi012.stdout:checking build system type... x86_64-pc-linux-gnu 2023-07-08T12:40:03.380 INFO:teuthology.orchestra.run.smithi012.stdout:checking host system type... x86_64-pc-linux-gnu 2023-07-08T12:40:03.381 INFO:teuthology.orchestra.run.smithi012.stdout:checking how to print strings... printf 2023-07-08T12:40:03.382 INFO:teuthology.orchestra.run.smithi012.stdout:checking for gcc... gcc 2023-07-08T12:40:03.510 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether the C compiler works... yes 2023-07-08T12:40:03.510 INFO:teuthology.orchestra.run.smithi012.stdout:checking for C compiler default output file name... a.out 2023-07-08T12:40:03.539 INFO:teuthology.orchestra.run.smithi012.stdout:checking for suffix of executables... 2023-07-08T12:40:03.598 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether we are cross compiling... no 2023-07-08T12:40:03.616 INFO:teuthology.orchestra.run.smithi012.stdout:checking for suffix of object files... o 2023-07-08T12:40:03.633 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether we are using the GNU C compiler... yes 2023-07-08T12:40:03.650 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether gcc accepts -g... yes 2023-07-08T12:40:03.677 INFO:teuthology.orchestra.run.smithi012.stdout:checking for gcc option to accept ISO C89... none needed 2023-07-08T12:40:03.679 INFO:teuthology.orchestra.run.smithi012.stdout:checking for a sed that does not truncate output... /bin/sed 2023-07-08T12:40:03.681 INFO:teuthology.orchestra.run.smithi012.stdout:checking for grep that handles long lines and -e... /bin/grep 2023-07-08T12:40:03.683 INFO:teuthology.orchestra.run.smithi012.stdout:checking for egrep... /bin/grep -E 2023-07-08T12:40:03.684 INFO:teuthology.orchestra.run.smithi012.stdout:checking for fgrep... /bin/grep -F 2023-07-08T12:40:03.687 INFO:teuthology.orchestra.run.smithi012.stdout:checking for ld used by gcc... /bin/ld 2023-07-08T12:40:03.688 INFO:teuthology.orchestra.run.smithi012.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2023-07-08T12:40:03.698 INFO:teuthology.orchestra.run.smithi012.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2023-07-08T12:40:03.714 INFO:teuthology.orchestra.run.smithi012.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2023-07-08T12:40:03.715 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether ln -s works... yes 2023-07-08T12:40:03.723 INFO:teuthology.orchestra.run.smithi012.stdout:checking the maximum length of command line arguments... 1572864 2023-07-08T12:40:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2023-07-08T12:40:03.728 INFO:teuthology.orchestra.run.smithi012.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2023-07-08T12:40:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:checking for /bin/ld option to reload object files... -r 2023-07-08T12:40:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:checking for objdump... objdump 2023-07-08T12:40:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:checking how to recognize dependent libraries... pass_all 2023-07-08T12:40:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:checking for dlltool... no 2023-07-08T12:40:03.729 INFO:teuthology.orchestra.run.smithi012.stdout:checking how to associate runtime and link libraries... printf %s\n 2023-07-08T12:40:03.730 INFO:teuthology.orchestra.run.smithi012.stdout:checking for ar... ar 2023-07-08T12:40:03.759 INFO:teuthology.orchestra.run.smithi012.stdout:checking for archiver @FILE support... @ 2023-07-08T12:40:03.759 INFO:teuthology.orchestra.run.smithi012.stdout:checking for strip... strip 2023-07-08T12:40:03.760 INFO:teuthology.orchestra.run.smithi012.stdout:checking for ranlib... ranlib 2023-07-08T12:40:03.760 INFO:teuthology.orchestra.run.smithi012.stdout:checking for gawk... gawk 2023-07-08T12:40:03.812 INFO:teuthology.orchestra.run.smithi012.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2023-07-08T12:40:03.813 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sysroot... no 2023-07-08T12:40:03.816 INFO:teuthology.orchestra.run.smithi012.stdout:checking for a working dd... /bin/dd 2023-07-08T12:40:03.818 INFO:teuthology.orchestra.run.smithi012.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2023-07-08T12:40:03.831 INFO:teuthology.orchestra.run.smithi012.stdout:checking for mt... no 2023-07-08T12:40:03.833 INFO:teuthology.orchestra.run.smithi012.stdout:checking if : is a manifest tool... no 2023-07-08T12:40:03.867 INFO:teuthology.orchestra.run.smithi012.stdout:checking how to run the C preprocessor... gcc -E 2023-07-08T12:40:03.983 INFO:teuthology.orchestra.run.smithi012.stdout:checking for ANSI C header files... yes 2023-07-08T12:40:04.003 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/types.h... yes 2023-07-08T12:40:04.027 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/stat.h... yes 2023-07-08T12:40:04.051 INFO:teuthology.orchestra.run.smithi012.stdout:checking for stdlib.h... yes 2023-07-08T12:40:04.076 INFO:teuthology.orchestra.run.smithi012.stdout:checking for string.h... yes 2023-07-08T12:40:04.100 INFO:teuthology.orchestra.run.smithi012.stdout:checking for memory.h... yes 2023-07-08T12:40:04.125 INFO:teuthology.orchestra.run.smithi012.stdout:checking for strings.h... yes 2023-07-08T12:40:04.152 INFO:teuthology.orchestra.run.smithi012.stdout:checking for inttypes.h... yes 2023-07-08T12:40:04.176 INFO:teuthology.orchestra.run.smithi012.stdout:checking for stdint.h... yes 2023-07-08T12:40:04.205 INFO:teuthology.orchestra.run.smithi012.stdout:checking for unistd.h... yes 2023-07-08T12:40:04.239 INFO:teuthology.orchestra.run.smithi012.stdout:checking for dlfcn.h... yes 2023-07-08T12:40:04.242 INFO:teuthology.orchestra.run.smithi012.stdout:checking for objdir... .libs 2023-07-08T12:40:04.309 INFO:teuthology.orchestra.run.smithi012.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2023-07-08T12:40:04.309 INFO:teuthology.orchestra.run.smithi012.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2023-07-08T12:40:04.324 INFO:teuthology.orchestra.run.smithi012.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2023-07-08T12:40:04.343 INFO:teuthology.orchestra.run.smithi012.stdout:checking if gcc static flag -static works... no 2023-07-08T12:40:04.364 INFO:teuthology.orchestra.run.smithi012.stdout:checking if gcc supports -c -o file.o... yes 2023-07-08T12:40:04.364 INFO:teuthology.orchestra.run.smithi012.stdout:checking if gcc supports -c -o file.o... (cached) yes 2023-07-08T12:40:04.370 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2023-07-08T12:40:04.395 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether -lc should be explicitly linked in... no 2023-07-08T12:40:04.439 INFO:teuthology.orchestra.run.smithi012.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2023-07-08T12:40:04.439 INFO:teuthology.orchestra.run.smithi012.stdout:checking how to hardcode library paths into programs... immediate 2023-07-08T12:40:04.441 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether stripping libraries is possible... yes 2023-07-08T12:40:04.441 INFO:teuthology.orchestra.run.smithi012.stdout:checking if libtool supports shared libraries... yes 2023-07-08T12:40:04.441 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether to build shared libraries... yes 2023-07-08T12:40:04.442 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether to build static libraries... yes 2023-07-08T12:40:04.446 INFO:teuthology.orchestra.run.smithi012.stdout:checking for gcc... (cached) gcc 2023-07-08T12:40:04.461 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-07-08T12:40:04.462 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether gcc accepts -g... (cached) yes 2023-07-08T12:40:04.462 INFO:teuthology.orchestra.run.smithi012.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-07-08T12:40:04.462 INFO:teuthology.orchestra.run.smithi012.stdout:checking for gmake... /usr/bin/gmake 2023-07-08T12:40:04.462 INFO:teuthology.orchestra.run.smithi012.stderr:./configure: line 12070: test: $(SHELL): binary operator expected 2023-07-08T12:40:04.463 INFO:teuthology.orchestra.run.smithi012.stdout:checking for tar... /bin/tar 2023-07-08T12:40:04.463 INFO:teuthology.orchestra.run.smithi012.stdout:checking for gzip... /bin/gzip 2023-07-08T12:40:04.469 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether gcc -MM is supported... yes 2023-07-08T12:40:04.470 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sort... /bin/sort 2023-07-08T12:40:04.470 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether ln -s works... yes 2023-07-08T12:40:04.471 INFO:teuthology.orchestra.run.smithi012.stdout:checking for rpm... /bin/rpm 2023-07-08T12:40:04.474 INFO:teuthology.orchestra.run.smithi012.stdout:checking for rpmbuild... no 2023-07-08T12:40:04.499 INFO:teuthology.orchestra.run.smithi012.stdout:checking assert.h usability... yes 2023-07-08T12:40:04.506 INFO:teuthology.orchestra.run.smithi012.stdout:checking assert.h presence... yes 2023-07-08T12:40:04.506 INFO:teuthology.orchestra.run.smithi012.stdout:checking for assert.h... yes 2023-07-08T12:40:04.532 INFO:teuthology.orchestra.run.smithi012.stdout:checking bstring.h usability... no 2023-07-08T12:40:04.542 INFO:teuthology.orchestra.run.smithi012.stdout:checking bstring.h presence... no 2023-07-08T12:40:04.542 INFO:teuthology.orchestra.run.smithi012.stdout:checking for bstring.h... no 2023-07-08T12:40:04.567 INFO:teuthology.orchestra.run.smithi012.stdout:checking libgen.h usability... yes 2023-07-08T12:40:04.574 INFO:teuthology.orchestra.run.smithi012.stdout:checking libgen.h presence... yes 2023-07-08T12:40:04.574 INFO:teuthology.orchestra.run.smithi012.stdout:checking for libgen.h... yes 2023-07-08T12:40:04.602 INFO:teuthology.orchestra.run.smithi012.stdout:checking dirent.h usability... yes 2023-07-08T12:40:04.609 INFO:teuthology.orchestra.run.smithi012.stdout:checking dirent.h presence... yes 2023-07-08T12:40:04.610 INFO:teuthology.orchestra.run.smithi012.stdout:checking for dirent.h... yes 2023-07-08T12:40:04.661 INFO:teuthology.orchestra.run.smithi012.stdout:checking errno.h usability... yes 2023-07-08T12:40:04.668 INFO:teuthology.orchestra.run.smithi012.stdout:checking errno.h presence... yes 2023-07-08T12:40:04.668 INFO:teuthology.orchestra.run.smithi012.stdout:checking for errno.h... yes 2023-07-08T12:40:04.696 INFO:teuthology.orchestra.run.smithi012.stdout:checking malloc.h usability... yes 2023-07-08T12:40:04.704 INFO:teuthology.orchestra.run.smithi012.stdout:checking malloc.h presence... yes 2023-07-08T12:40:04.705 INFO:teuthology.orchestra.run.smithi012.stdout:checking for malloc.h... yes 2023-07-08T12:40:04.731 INFO:teuthology.orchestra.run.smithi012.stdout:checking uuid.h usability... no 2023-07-08T12:40:04.741 INFO:teuthology.orchestra.run.smithi012.stdout:checking uuid.h presence... no 2023-07-08T12:40:04.741 INFO:teuthology.orchestra.run.smithi012.stdout:checking for uuid.h... no 2023-07-08T12:40:04.767 INFO:teuthology.orchestra.run.smithi012.stdout:checking uuid/uuid.h usability... yes 2023-07-08T12:40:04.776 INFO:teuthology.orchestra.run.smithi012.stdout:checking uuid/uuid.h presence... yes 2023-07-08T12:40:04.776 INFO:teuthology.orchestra.run.smithi012.stdout:checking for uuid/uuid.h... yes 2023-07-08T12:40:04.802 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/uuid.h usability... no 2023-07-08T12:40:04.812 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/uuid.h presence... no 2023-07-08T12:40:04.812 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/uuid.h... no 2023-07-08T12:40:04.838 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/file.h usability... yes 2023-07-08T12:40:04.845 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/file.h presence... yes 2023-07-08T12:40:04.846 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/file.h... yes 2023-07-08T12:40:04.873 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/fcntl.h usability... yes 2023-07-08T12:40:04.881 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/fcntl.h presence... yes 2023-07-08T12:40:04.881 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/fcntl.h... yes 2023-07-08T12:40:04.907 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/syssgi.h usability... no 2023-07-08T12:40:04.916 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/syssgi.h presence... no 2023-07-08T12:40:04.916 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/syssgi.h... no 2023-07-08T12:40:04.961 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/param.h usability... yes 2023-07-08T12:40:04.970 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/param.h presence... yes 2023-07-08T12:40:04.971 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/param.h... yes 2023-07-08T12:40:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/stat.h... (cached) yes 2023-07-08T12:40:05.003 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/statvfs.h usability... yes 2023-07-08T12:40:05.010 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/statvfs.h presence... yes 2023-07-08T12:40:05.011 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/statvfs.h... yes 2023-07-08T12:40:05.037 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/time.h usability... yes 2023-07-08T12:40:05.045 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/time.h presence... yes 2023-07-08T12:40:05.045 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/time.h... yes 2023-07-08T12:40:05.104 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/ioctl.h usability... yes 2023-07-08T12:40:05.111 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/ioctl.h presence... yes 2023-07-08T12:40:05.111 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/ioctl.h... yes 2023-07-08T12:40:05.141 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/wait.h usability... yes 2023-07-08T12:40:05.150 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/wait.h presence... yes 2023-07-08T12:40:05.150 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/wait.h... yes 2023-07-08T12:40:05.155 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/types.h... (cached) yes 2023-07-08T12:40:05.160 INFO:teuthology.orchestra.run.smithi012.stdout:checking for strings.h... (cached) yes 2023-07-08T12:40:05.188 INFO:teuthology.orchestra.run.smithi012.stdout:checking err.h usability... yes 2023-07-08T12:40:05.194 INFO:teuthology.orchestra.run.smithi012.stdout:checking err.h presence... yes 2023-07-08T12:40:05.194 INFO:teuthology.orchestra.run.smithi012.stdout:checking for err.h... yes 2023-07-08T12:40:05.238 INFO:teuthology.orchestra.run.smithi012.stdout:checking linux/falloc.h usability... yes 2023-07-08T12:40:05.244 INFO:teuthology.orchestra.run.smithi012.stdout:checking linux/falloc.h presence... yes 2023-07-08T12:40:05.245 INFO:teuthology.orchestra.run.smithi012.stdout:checking for linux/falloc.h... yes 2023-07-08T12:40:05.271 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/fs/xfs_fsops.h usability... no 2023-07-08T12:40:05.280 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/fs/xfs_fsops.h presence... no 2023-07-08T12:40:05.280 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/fs/xfs_fsops.h... no 2023-07-08T12:40:05.304 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/fs/xfs_itable.h usability... no 2023-07-08T12:40:05.313 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/fs/xfs_itable.h presence... no 2023-07-08T12:40:05.313 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/fs/xfs_itable.h... no 2023-07-08T12:40:05.337 INFO:teuthology.orchestra.run.smithi012.stdout:checking xfs/platform_defs.h usability... no 2023-07-08T12:40:05.346 INFO:teuthology.orchestra.run.smithi012.stdout:checking xfs/platform_defs.h presence... no 2023-07-08T12:40:05.347 INFO:teuthology.orchestra.run.smithi012.stdout:checking for xfs/platform_defs.h... no 2023-07-08T12:40:05.370 INFO:teuthology.orchestra.run.smithi012.stdout:checking btrfs/ioctl.h usability... no 2023-07-08T12:40:05.379 INFO:teuthology.orchestra.run.smithi012.stdout:checking btrfs/ioctl.h presence... no 2023-07-08T12:40:05.380 INFO:teuthology.orchestra.run.smithi012.stdout:checking for btrfs/ioctl.h... no 2023-07-08T12:40:05.403 INFO:teuthology.orchestra.run.smithi012.stdout:checking cifs/ioctl.h usability... no 2023-07-08T12:40:05.412 INFO:teuthology.orchestra.run.smithi012.stdout:checking cifs/ioctl.h presence... no 2023-07-08T12:40:05.413 INFO:teuthology.orchestra.run.smithi012.stdout:checking for cifs/ioctl.h... no 2023-07-08T12:40:05.439 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/mman.h usability... yes 2023-07-08T12:40:05.446 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/mman.h presence... yes 2023-07-08T12:40:05.446 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/mman.h... yes 2023-07-08T12:40:05.461 INFO:teuthology.orchestra.run.smithi012.stdout:checking for xfs/xfs_log_format.h... no 2023-07-08T12:40:05.495 INFO:teuthology.orchestra.run.smithi012.stdout:checking for uuid_compare... no 2023-07-08T12:40:05.526 INFO:teuthology.orchestra.run.smithi012.stdout:checking for uuid_compare in -luuid... yes 2023-07-08T12:40:05.574 INFO:teuthology.orchestra.run.smithi012.stdout:checking for xfs/xfs.h... yes 2023-07-08T12:40:05.587 INFO:teuthology.orchestra.run.smithi012.stdout:checking for xfs/libxfs.h... no 2023-07-08T12:40:05.602 INFO:teuthology.orchestra.run.smithi012.stdout:checking whether xlog_assign_lsn is declared... no 2023-07-08T12:40:05.630 INFO:teuthology.orchestra.run.smithi012.stdout:checking for xfs/xqm.h... yes 2023-07-08T12:40:05.676 INFO:teuthology.orchestra.run.smithi012.stdout:checking xfsctl from xfs/xfs.h... ok 2023-07-08T12:40:05.699 INFO:teuthology.orchestra.run.smithi012.stdout:checking xfs/handle.h usability... yes 2023-07-08T12:40:05.705 INFO:teuthology.orchestra.run.smithi012.stdout:checking xfs/handle.h presence... yes 2023-07-08T12:40:05.705 INFO:teuthology.orchestra.run.smithi012.stdout:checking for xfs/handle.h... yes 2023-07-08T12:40:05.744 INFO:teuthology.orchestra.run.smithi012.stdout:checking for attr_list_by_handle in -lhandle... yes 2023-07-08T12:40:05.775 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/acl.h usability... yes 2023-07-08T12:40:05.782 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/acl.h presence... yes 2023-07-08T12:40:05.783 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/acl.h... yes 2023-07-08T12:40:05.819 INFO:teuthology.orchestra.run.smithi012.stdout:checking acl/libacl.h usability... yes 2023-07-08T12:40:05.827 INFO:teuthology.orchestra.run.smithi012.stdout:checking acl/libacl.h presence... yes 2023-07-08T12:40:05.827 INFO:teuthology.orchestra.run.smithi012.stdout:checking for acl/libacl.h... yes 2023-07-08T12:40:05.859 INFO:teuthology.orchestra.run.smithi012.stdout:checking for acl_init in -lacl... yes 2023-07-08T12:40:05.882 INFO:teuthology.orchestra.run.smithi012.stdout:checking gdbm-ndbm.h usability... no 2023-07-08T12:40:05.892 INFO:teuthology.orchestra.run.smithi012.stdout:checking gdbm-ndbm.h presence... no 2023-07-08T12:40:05.892 INFO:teuthology.orchestra.run.smithi012.stdout:checking for gdbm-ndbm.h... no 2023-07-08T12:40:05.913 INFO:teuthology.orchestra.run.smithi012.stdout:checking gdbm/ndbm.h usability... no 2023-07-08T12:40:05.922 INFO:teuthology.orchestra.run.smithi012.stdout:checking gdbm/ndbm.h presence... no 2023-07-08T12:40:05.923 INFO:teuthology.orchestra.run.smithi012.stdout:checking for gdbm/ndbm.h... no 2023-07-08T12:40:05.944 INFO:teuthology.orchestra.run.smithi012.stdout:checking ndbm.h usability... no 2023-07-08T12:40:05.954 INFO:teuthology.orchestra.run.smithi012.stdout:checking ndbm.h presence... no 2023-07-08T12:40:05.954 INFO:teuthology.orchestra.run.smithi012.stdout:checking for ndbm.h... no 2023-07-08T12:40:05.979 INFO:teuthology.orchestra.run.smithi012.stdout:checking libaio.h usability... yes 2023-07-08T12:40:05.989 INFO:teuthology.orchestra.run.smithi012.stdout:checking libaio.h presence... yes 2023-07-08T12:40:05.990 INFO:teuthology.orchestra.run.smithi012.stdout:checking for libaio.h... yes 2023-07-08T12:40:06.012 INFO:teuthology.orchestra.run.smithi012.stdout:checking liburing.h usability... no 2023-07-08T12:40:06.021 INFO:teuthology.orchestra.run.smithi012.stdout:checking liburing.h presence... no 2023-07-08T12:40:06.022 INFO:teuthology.orchestra.run.smithi012.stdout:checking for liburing.h... no 2023-07-08T12:40:06.043 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/capability.h usability... no 2023-07-08T12:40:06.053 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/capability.h presence... no 2023-07-08T12:40:06.053 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/capability.h... no 2023-07-08T12:40:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:checking linux/fiemap.h usability... yes 2023-07-08T12:40:06.083 INFO:teuthology.orchestra.run.smithi012.stdout:checking linux/fiemap.h presence... yes 2023-07-08T12:40:06.083 INFO:teuthology.orchestra.run.smithi012.stdout:checking for linux/fiemap.h... yes 2023-07-08T12:40:06.116 INFO:teuthology.orchestra.run.smithi012.stdout:checking for fallocate... yes 2023-07-08T12:40:06.149 INFO:teuthology.orchestra.run.smithi012.stdout:checking for open_by_handle_at... yes 2023-07-08T12:40:06.173 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/prctl.h usability... yes 2023-07-08T12:40:06.181 INFO:teuthology.orchestra.run.smithi012.stdout:checking sys/prctl.h presence... yes 2023-07-08T12:40:06.181 INFO:teuthology.orchestra.run.smithi012.stdout:checking for sys/prctl.h... yes 2023-07-08T12:40:06.205 INFO:teuthology.orchestra.run.smithi012.stdout:checking linux/fs.h usability... yes 2023-07-08T12:40:06.212 INFO:teuthology.orchestra.run.smithi012.stdout:checking linux/fs.h presence... yes 2023-07-08T12:40:06.212 INFO:teuthology.orchestra.run.smithi012.stdout:checking for linux/fs.h... yes 2023-07-08T12:40:06.233 INFO:teuthology.orchestra.run.smithi012.stdout:checking btrfsutil.h usability... no 2023-07-08T12:40:06.243 INFO:teuthology.orchestra.run.smithi012.stdout:checking btrfsutil.h presence... no 2023-07-08T12:40:06.243 INFO:teuthology.orchestra.run.smithi012.stdout:checking for btrfsutil.h... no 2023-07-08T12:40:06.262 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:40:06.264 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:40:06.278 INFO:teuthology.orchestra.run.smithi012.stdout:checking for copy_file_range... yes 2023-07-08T12:40:06.314 INFO:teuthology.orchestra.run.smithi012.stdout:checking for SEEK_DATA... yes 2023-07-08T12:40:06.360 INFO:teuthology.orchestra.run.smithi012.stdout:checking for BMV_OF_SHARED... yes 2023-07-08T12:40:06.396 INFO:teuthology.orchestra.run.smithi012.stdout:checking for nftw... yes 2023-07-08T12:40:06.431 INFO:teuthology.orchestra.run.smithi012.stdout:checking for RLIMIT_NOFILE... yes 2023-07-08T12:40:06.448 INFO:teuthology.orchestra.run.smithi012.stdout:checking for FIEXCHANGE_RANGE... no 2023-07-08T12:40:06.481 INFO:teuthology.orchestra.run.smithi012.stdout:checking for renameat2... yes 2023-07-08T12:40:06.514 INFO:teuthology.orchestra.run.smithi012.stdout:checking for reallocarray... yes 2023-07-08T12:40:06.528 INFO:teuthology.orchestra.run.smithi012.stdout:checking for struct mount_attr... no 2023-07-08T12:40:06.580 INFO:teuthology.orchestra.run.smithi012.stdout:checking for struct btrfs_qgroup_limit... yes 2023-07-08T12:40:06.614 INFO:teuthology.orchestra.run.smithi012.stdout:checking for struct btrfs_qgroup_inherit... yes 2023-07-08T12:40:06.649 INFO:teuthology.orchestra.run.smithi012.stdout:checking for struct btrfs_ioctl_vol_args... yes 2023-07-08T12:40:06.684 INFO:teuthology.orchestra.run.smithi012.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2023-07-08T12:40:06.718 INFO:teuthology.orchestra.run.smithi012.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2023-07-08T12:40:06.752 INFO:teuthology.orchestra.run.smithi012.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2023-07-08T12:40:06.787 INFO:teuthology.orchestra.run.smithi012.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2023-07-08T12:40:06.813 INFO:teuthology.orchestra.run.smithi012.stdout:checking linux/btrfs.h usability... yes 2023-07-08T12:40:06.820 INFO:teuthology.orchestra.run.smithi012.stdout:checking linux/btrfs.h presence... yes 2023-07-08T12:40:06.820 INFO:teuthology.orchestra.run.smithi012.stdout:checking for linux/btrfs.h... yes 2023-07-08T12:40:06.851 INFO:teuthology.orchestra.run.smithi012.stdout:checking linux/btrfs_tree.h usability... yes 2023-07-08T12:40:06.858 INFO:teuthology.orchestra.run.smithi012.stdout:checking linux/btrfs_tree.h presence... yes 2023-07-08T12:40:06.858 INFO:teuthology.orchestra.run.smithi012.stdout:checking for linux/btrfs_tree.h... yes 2023-07-08T12:40:06.889 INFO:teuthology.orchestra.run.smithi012.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... no 2023-07-08T12:40:06.906 INFO:teuthology.orchestra.run.smithi012.stdout:configure: creating ./config.status 2023-07-08T12:40:07.205 INFO:teuthology.orchestra.run.smithi012.stdout:config.status: creating include/builddefs 2023-07-08T12:40:07.215 INFO:teuthology.orchestra.run.smithi012.stdout:config.status: creating include/config.h 2023-07-08T12:40:07.223 INFO:teuthology.orchestra.run.smithi012.stdout:config.status: executing libtool commands 2023-07-08T12:40:07.261 INFO:teuthology.orchestra.run.smithi012.stdout:Building include 2023-07-08T12:40:07.264 INFO:teuthology.orchestra.run.smithi012.stdout:Building lib 2023-07-08T12:40:07.267 INFO:teuthology.orchestra.run.smithi012.stdout: [LTDEP] 2023-07-08T12:40:07.372 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] dataascii.lo 2023-07-08T12:40:07.486 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] databin.lo 2023-07-08T12:40:07.626 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] datapid.lo 2023-07-08T12:40:07.726 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] file_lock.lo 2023-07-08T12:40:07.867 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] forker.lo 2023-07-08T12:40:07.994 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] pattern.lo 2023-07-08T12:40:08.111 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] open_flags.lo 2023-07-08T12:40:08.246 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] random_range.lo 2023-07-08T12:40:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] string_to_tokens.lo 2023-07-08T12:40:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] str_to_bytes.lo 2023-07-08T12:40:08.686 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] tlibio.lo 2023-07-08T12:40:08.904 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] write_log.lo 2023-07-08T12:40:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] random.lo 2023-07-08T12:40:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: [LD] 2023-07-08T12:40:09.351 INFO:teuthology.orchestra.run.smithi012.stdout:Building ltp 2023-07-08T12:40:09.354 INFO:teuthology.orchestra.run.smithi012.stdout: [DEP] 2023-07-08T12:40:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] doio 2023-07-08T12:40:10.096 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] fsstress 2023-07-08T12:40:11.343 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] fsx 2023-07-08T12:40:12.159 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] growfiles 2023-07-08T12:40:12.638 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] iogen 2023-07-08T12:40:12.970 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-stress 2023-07-08T12:40:13.362 INFO:teuthology.orchestra.run.smithi012.stdout:Building src 2023-07-08T12:40:13.368 INFO:teuthology.orchestra.run.smithi012.stdout: [DEP] 2023-07-08T12:40:14.826 INFO:teuthology.orchestra.run.smithi012.stdout: [DEP] 2023-07-08T12:40:14.852 INFO:teuthology.orchestra.run.smithi012.stdout: [DEP] 2023-07-08T12:40:14.860 INFO:teuthology.orchestra.run.smithi012.stdout: [DEP] 2023-07-08T12:40:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: [DEP] 2023-07-08T12:40:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] dirstress 2023-07-08T12:40:15.289 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] fill 2023-07-08T12:40:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] fill2 2023-07-08T12:40:15.647 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] getpagesize 2023-07-08T12:40:15.775 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] holes 2023-07-08T12:40:15.956 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] lstat64 2023-07-08T12:40:16.128 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] nametest 2023-07-08T12:40:16.354 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] permname 2023-07-08T12:40:16.541 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] randholes 2023-07-08T12:40:16.772 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] runas 2023-07-08T12:40:16.936 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] truncfile 2023-07-08T12:40:17.103 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] usemem 2023-07-08T12:40:17.244 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] mmapcat 2023-07-08T12:40:17.384 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] append_reader 2023-07-08T12:40:17.526 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] append_writer 2023-07-08T12:40:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] dirperf 2023-07-08T12:40:17.874 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] metaperf 2023-07-08T12:40:18.111 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] devzero 2023-07-08T12:40:18.274 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] feature 2023-07-08T12:40:18.476 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] alloc 2023-07-08T12:40:18.674 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] fault 2023-07-08T12:40:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] fstest 2023-07-08T12:40:19.048 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_access_root 2023-07-08T12:40:19.258 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] godown 2023-07-08T12:40:19.468 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] resvtest 2023-07-08T12:40:19.687 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] writemod 2023-07-08T12:40:19.825 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] writev_on_pagefault 2023-07-08T12:40:19.976 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] makeextents 2023-07-08T12:40:20.143 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] itrash 2023-07-08T12:40:20.289 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] rename 2023-07-08T12:40:20.420 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] multi_open_unlink 2023-07-08T12:40:20.575 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] unwritten_sync 2023-07-08T12:40:20.743 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] genhashnames 2023-07-08T12:40:20.916 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_holes 2023-07-08T12:40:21.054 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_mmap_writev 2023-07-08T12:40:21.193 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_truncate_cmtime 2023-07-08T12:40:21.358 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] dirhash_collide 2023-07-08T12:40:21.528 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_rename_overwrite 2023-07-08T12:40:21.661 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] holetest 2023-07-08T12:40:21.864 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_truncate_self 2023-07-08T12:40:21.997 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] af_unix 2023-07-08T12:40:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_mmap_stale_pmd 2023-07-08T12:40:22.284 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_mmap_cow_race 2023-07-08T12:40:22.440 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_mmap_fallocate 2023-07-08T12:40:22.582 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] fsync-err 2023-07-08T12:40:22.755 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_mmap_write_ro 2023-07-08T12:40:22.896 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_ext4_dax_journal_corruption 2023-07-08T12:40:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_ext4_dax_inline_corruption 2023-07-08T12:40:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_ofd_locks 2023-07-08T12:40:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_mmap_collision 2023-07-08T12:40:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] mmap-write-concurrent 2023-07-08T12:40:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_get_file_time 2023-07-08T12:40:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_create_short_dirs 2023-07-08T12:40:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_create_long_dirs 2023-07-08T12:40:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_enospc 2023-07-08T12:40:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_mmap_writev_overlap 2023-07-08T12:40:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] checkpoint_journal 2023-07-08T12:40:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] mmap-rw-fault 2023-07-08T12:40:24.826 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] allocstale 2023-07-08T12:40:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_mmap_cow_memory_failure 2023-07-08T12:40:25.142 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] fake-dump-rootino 2023-07-08T12:40:25.282 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] dio-buf-fault 2023-07-08T12:40:25.427 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] xfsctl 2023-07-08T12:40:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] bstat 2023-07-08T12:40:25.794 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_mtab 2023-07-08T12:40:25.966 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] getdevicesize 2023-07-08T12:40:26.101 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] preallo_rw_pattern_reader 2023-07-08T12:40:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] preallo_rw_pattern_writer 2023-07-08T12:40:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] ftrunc 2023-07-08T12:40:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] trunc 2023-07-08T12:40:26.691 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] fs_perms 2023-07-08T12:40:26.845 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] testx 2023-07-08T12:40:26.971 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] looptest 2023-07-08T12:40:27.153 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] locktest 2023-07-08T12:40:27.440 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] unwritten_mmap 2023-07-08T12:40:27.592 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] bulkstat_unlink_test 2023-07-08T12:40:27.779 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] deduperace 2023-07-08T12:40:27.962 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] bulkstat_unlink_test_modified 2023-07-08T12:40:28.147 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_dir_offset 2023-07-08T12:40:28.287 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_futimens 2023-07-08T12:40:28.420 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_immutable 2023-07-08T12:40:28.989 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] stale_handle 2023-07-08T12:40:29.148 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] pwrite_mmap_blocked 2023-07-08T12:40:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_dir_offset2 2023-07-08T12:40:29.471 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] seek_sanity_test 2023-07-08T12:40:29.860 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] seek_copy_test 2023-07-08T12:40:30.070 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_readdir_1 2023-07-08T12:40:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_readdir_2 2023-07-08T12:40:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] fsync-tester 2023-07-08T12:40:30.657 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] nsexec 2023-07-08T12:40:30.822 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] cloner 2023-07-08T12:40:30.988 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] renameat2 2023-07-08T12:40:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_getcwd 2023-07-08T12:40:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] e4compact 2023-07-08T12:40:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] test-nextquota 2023-07-08T12:40:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] punch-alternating 2023-07-08T12:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] attr-list-by-handle-cursor-test 2023-07-08T12:40:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] listxattr 2023-07-08T12:40:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] dio-interleaved 2023-07-08T12:40:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_dir_type 2023-07-08T12:40:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] dio-invalidate-cache 2023-07-08T12:40:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] stat_test 2023-07-08T12:40:32.860 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_encrypted_d_revalidate 2023-07-08T12:40:33.016 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] attr_replace_test 2023-07-08T12:40:33.169 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] swapon 2023-07-08T12:40:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] mkswap 2023-07-08T12:40:33.471 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_attr_corruption 2023-07-08T12:40:33.612 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_open_tmpfiles 2023-07-08T12:40:33.780 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] fscrypt-crypt-util 2023-07-08T12:40:34.378 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] bulkstat_null_ocount 2023-07-08T12:40:34.528 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] splice-test 2023-07-08T12:40:34.698 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] chprojid_fail 2023-07-08T12:40:34.851 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] detached_mounts_propagation 2023-07-08T12:40:35.024 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] ext4_resize 2023-07-08T12:40:35.165 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_readdir_3 2023-07-08T12:40:35.348 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] splice2pipe 2023-07-08T12:40:35.502 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] uuid_ioctl 2023-07-08T12:40:35.649 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] fiemap-tester 2023-07-08T12:40:35.900 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_stripealign 2023-07-08T12:40:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] open_by_handle 2023-07-08T12:40:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] t_mmap_dio 2023-07-08T12:40:36.341 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:40:36.343 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:40:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] xfsfind 2023-07-08T12:40:36.620 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] fssum 2023-07-08T12:40:37.023 INFO:teuthology.orchestra.run.smithi012.stdout:Building log-writes 2023-07-08T12:40:37.028 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] replay-log 2023-07-08T12:40:37.319 INFO:teuthology.orchestra.run.smithi012.stdout:Building perf 2023-07-08T12:40:37.322 INFO:teuthology.orchestra.run.smithi012.stdout:Building aio-dio-regress 2023-07-08T12:40:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-dio-append-write-fallocate-race 2023-07-08T12:40:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-dio-fcntl-race 2023-07-08T12:40:37.649 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-dio-subblock-eof-read 2023-07-08T12:40:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-dio-append-write-read-race 2023-07-08T12:40:37.955 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-dio-invalidate-readahead 2023-07-08T12:40:38.113 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-dio-write-verify 2023-07-08T12:40:38.297 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2023-07-08T12:40:38.429 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-last-ref-held-by-io 2023-07-08T12:40:38.596 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-dio-cycle-write 2023-07-08T12:40:38.773 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-dio-extend-stat 2023-07-08T12:40:38.932 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2023-07-08T12:40:39.061 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-dio-hole-filling-race 2023-07-08T12:40:39.205 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-dio-invalidate-failure 2023-07-08T12:40:39.358 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aiodio_sparse2 2023-07-08T12:40:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aiocp 2023-07-08T12:40:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-dio-cow-race 2023-07-08T12:40:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] aio-dio-eof-race 2023-07-08T12:40:40.103 INFO:teuthology.orchestra.run.smithi012.stdout:Building vfs 2023-07-08T12:40:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] vfstest 2023-07-08T12:40:41.855 INFO:teuthology.orchestra.run.smithi012.stdout: [CC] mount-idmapped 2023-07-08T12:40:42.200 INFO:teuthology.orchestra.run.smithi012.stdout:Building m4 2023-07-08T12:40:42.204 INFO:teuthology.orchestra.run.smithi012.stdout:Building common 2023-07-08T12:40:42.207 INFO:teuthology.orchestra.run.smithi012.stdout:Building tests 2023-07-08T12:40:42.212 INFO:teuthology.orchestra.run.smithi012.stdout:Building perf 2023-07-08T12:40:42.217 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/perf/group.list 2023-07-08T12:40:42.228 INFO:teuthology.orchestra.run.smithi012.stdout:Building ext4 2023-07-08T12:40:42.233 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/ext4/group.list 2023-07-08T12:40:42.267 INFO:teuthology.orchestra.run.smithi012.stdout:Building ocfs2 2023-07-08T12:40:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/ocfs2/group.list 2023-07-08T12:40:42.285 INFO:teuthology.orchestra.run.smithi012.stdout:Building overlay 2023-07-08T12:40:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/overlay/group.list 2023-07-08T12:40:42.325 INFO:teuthology.orchestra.run.smithi012.stdout:Building cifs 2023-07-08T12:40:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/cifs/group.list 2023-07-08T12:40:42.342 INFO:teuthology.orchestra.run.smithi012.stdout:Building generic 2023-07-08T12:40:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/generic/group.list 2023-07-08T12:40:42.440 INFO:teuthology.orchestra.run.smithi012.stdout:Building shared 2023-07-08T12:40:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/shared/group.list 2023-07-08T12:40:42.461 INFO:teuthology.orchestra.run.smithi012.stdout:Building nfs 2023-07-08T12:40:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/nfs/group.list 2023-07-08T12:40:42.479 INFO:teuthology.orchestra.run.smithi012.stdout:Building f2fs 2023-07-08T12:40:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/f2fs/group.list 2023-07-08T12:40:42.499 INFO:teuthology.orchestra.run.smithi012.stdout:Building udf 2023-07-08T12:40:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/udf/group.list 2023-07-08T12:40:42.515 INFO:teuthology.orchestra.run.smithi012.stdout:Building ceph 2023-07-08T12:40:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/ceph/group.list 2023-07-08T12:40:42.533 INFO:teuthology.orchestra.run.smithi012.stdout:Building xfs 2023-07-08T12:40:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/xfs/group.list 2023-07-08T12:40:42.633 INFO:teuthology.orchestra.run.smithi012.stdout:Building btrfs 2023-07-08T12:40:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/btrfs/group.list 2023-07-08T12:40:42.694 INFO:teuthology.orchestra.run.smithi012.stdout:Building tmpfs 2023-07-08T12:40:42.699 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/tmpfs/group.list 2023-07-08T12:40:42.711 INFO:teuthology.orchestra.run.smithi012.stdout:Building selftest 2023-07-08T12:40:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/selftest/group.list 2023-07-08T12:40:42.729 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo make install) 2023-07-08T12:40:42.810 INFO:teuthology.orchestra.run.smithi012.stdout:Building include 2023-07-08T12:40:42.814 INFO:teuthology.orchestra.run.smithi012.stdout:Building lib 2023-07-08T12:40:42.818 INFO:teuthology.orchestra.run.smithi012.stdout:Building ltp 2023-07-08T12:40:42.822 INFO:teuthology.orchestra.run.smithi012.stdout:Building src 2023-07-08T12:40:42.855 INFO:teuthology.orchestra.run.smithi012.stdout:Building log-writes 2023-07-08T12:40:42.858 INFO:teuthology.orchestra.run.smithi012.stdout:Building perf 2023-07-08T12:40:42.861 INFO:teuthology.orchestra.run.smithi012.stdout:Building aio-dio-regress 2023-07-08T12:40:42.865 INFO:teuthology.orchestra.run.smithi012.stdout:Building vfs 2023-07-08T12:40:42.869 INFO:teuthology.orchestra.run.smithi012.stdout:Building m4 2023-07-08T12:40:42.872 INFO:teuthology.orchestra.run.smithi012.stdout:Building common 2023-07-08T12:40:42.876 INFO:teuthology.orchestra.run.smithi012.stdout:Building tests 2023-07-08T12:40:42.881 INFO:teuthology.orchestra.run.smithi012.stdout:Building perf 2023-07-08T12:40:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/perf/group.list 2023-07-08T12:40:42.896 INFO:teuthology.orchestra.run.smithi012.stdout:Building ext4 2023-07-08T12:40:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/ext4/group.list 2023-07-08T12:40:42.936 INFO:teuthology.orchestra.run.smithi012.stdout:Building ocfs2 2023-07-08T12:40:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/ocfs2/group.list 2023-07-08T12:40:42.953 INFO:teuthology.orchestra.run.smithi012.stdout:Building overlay 2023-07-08T12:40:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/overlay/group.list 2023-07-08T12:40:42.993 INFO:teuthology.orchestra.run.smithi012.stdout:Building cifs 2023-07-08T12:40:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/cifs/group.list 2023-07-08T12:40:43.010 INFO:teuthology.orchestra.run.smithi012.stdout:Building generic 2023-07-08T12:40:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/generic/group.list 2023-07-08T12:40:43.107 INFO:teuthology.orchestra.run.smithi012.stdout:Building shared 2023-07-08T12:40:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/shared/group.list 2023-07-08T12:40:43.127 INFO:teuthology.orchestra.run.smithi012.stdout:Building nfs 2023-07-08T12:40:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/nfs/group.list 2023-07-08T12:40:43.145 INFO:teuthology.orchestra.run.smithi012.stdout:Building f2fs 2023-07-08T12:40:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/f2fs/group.list 2023-07-08T12:40:43.166 INFO:teuthology.orchestra.run.smithi012.stdout:Building udf 2023-07-08T12:40:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/udf/group.list 2023-07-08T12:40:43.181 INFO:teuthology.orchestra.run.smithi012.stdout:Building ceph 2023-07-08T12:40:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/ceph/group.list 2023-07-08T12:40:43.199 INFO:teuthology.orchestra.run.smithi012.stdout:Building xfs 2023-07-08T12:40:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/xfs/group.list 2023-07-08T12:40:43.297 INFO:teuthology.orchestra.run.smithi012.stdout:Building btrfs 2023-07-08T12:40:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/btrfs/group.list 2023-07-08T12:40:43.357 INFO:teuthology.orchestra.run.smithi012.stdout:Building tmpfs 2023-07-08T12:40:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/tmpfs/group.list 2023-07-08T12:40:43.374 INFO:teuthology.orchestra.run.smithi012.stdout:Building selftest 2023-07-08T12:40:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: [GROUP] /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/selftest/group.list 2023-07-08T12:40:43.390 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2023-07-08T12:40:43.392 INFO:teuthology.orchestra.run.smithi012.stdout:gmake[1]: Nothing to be done for 'install'. 2023-07-08T12:40:43.392 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2023-07-08T12:40:43.394 INFO:teuthology.orchestra.run.smithi012.stdout:gmake[1]: Nothing to be done for 'install'. 2023-07-08T12:40:43.394 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2023-07-08T12:40:43.396 INFO:teuthology.orchestra.run.smithi012.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2023-07-08T12:40:43.405 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2023-07-08T12:40:43.484 INFO:teuthology.orchestra.run.smithi012.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2023-07-08T12:40:43.493 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2023-07-08T12:40:43.510 INFO:teuthology.orchestra.run.smithi012.stdout:Building log-writes 2023-07-08T12:40:43.513 INFO:teuthology.orchestra.run.smithi012.stdout:Building perf 2023-07-08T12:40:43.516 INFO:teuthology.orchestra.run.smithi012.stdout:Building aio-dio-regress 2023-07-08T12:40:43.520 INFO:teuthology.orchestra.run.smithi012.stdout:Building vfs 2023-07-08T12:40:43.523 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake -C log-writes install 2023-07-08T12:40:43.525 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2023-07-08T12:40:43.534 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2023-07-08T12:40:43.543 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake -C perf install 2023-07-08T12:40:43.544 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2023-07-08T12:40:43.553 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql /var/lib/xfstests/src/perf 2023-07-08T12:40:43.576 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake -C aio-dio-regress install 2023-07-08T12:40:43.578 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2023-07-08T12:40:43.587 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 aio-dio-append-write-fallocate-race aio-dio-fcntl-race aio-dio-subblock-eof-read aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-last-ref-held-by-io aio-dio-cycle-write aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer aio-dio-hole-filling-race aio-dio-invalidate-failure aiodio_sparse2 aiocp aio-dio-cow-race aio-dio-eof-race /var/lib/xfstests/src/aio-dio-regress 2023-07-08T12:40:43.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake -C vfs install 2023-07-08T12:40:43.643 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2023-07-08T12:40:43.652 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2023-07-08T12:40:43.664 INFO:teuthology.orchestra.run.smithi012.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2023-07-08T12:40:43.673 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl fiemap-tester t_stripealign open_by_handle t_mmap_dio xfsfind fssum /var/lib/xfstests/src 2023-07-08T12:40:44.903 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py /var/lib/xfstests/src 2023-07-08T12:40:44.999 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2023-07-08T12:40:45.031 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2023-07-08T12:40:45.032 INFO:teuthology.orchestra.run.smithi012.stdout:gmake[1]: Nothing to be done for 'install'. 2023-07-08T12:40:45.032 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2023-07-08T12:40:45.034 INFO:teuthology.orchestra.run.smithi012.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2023-07-08T12:40:45.042 INFO:teuthology.orchestra.run.smithi012.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2023-07-08T12:40:45.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2023-07-08T12:40:45.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/btrfs/ install 2023-07-08T12:40:45.182 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2023-07-08T12:40:45.191 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 290 291 299 /var/lib/xfstests/tests/btrfs 2023-07-08T12:40:46.014 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2023-07-08T12:40:46.024 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 057.out 175.out 186.out 240.out 069.out 092.out 077.out 140.out 265.out 236.out 108.out 204.out 177.out 194.out 234.out 249.out 109.out 033.out 082.out 252.out 270.out 040.out 176.out 202.out 137.out 034.out 125.out 014.out 165.out 181.out 152.out 142.out 264.out 192.out 148.out 282.out 267.out 052.out 228.out 183.out 094.out 273.out 251.out 110.out 043.out 209.out 151.out 206.out 195.out 274.out 215.out 115.out 015.out 250.out 191.out 281.out 046.out 051.out 244.out 219.out 119.out 019.out 128.out 232.out 248.out 162.out 147.out 197.out 196.out 242.out 004.out 189.out 220.out 120.out 020.out 102.out 263.out 171.out 261.out 045.out 217.out 117.out 017.out 072.out 068.out 048.out 062.out 038.out 084.out 129.out 233.out 066.out 029.out 229.out 103.out 254.out 111.out 011.out 243.out 275.out 134.out 170.out 260.out 157.out 143.out 133.out 286.out 161.out 246.out 216.out 116.out 016.out 155.out 118.out 287.out 067.out 010.out 238.out 138.out 095.out 184.out 160.out 190.out 291.out 081.out 024.out 071.out 061.out 047.out 200.out 100.out 025.out 139.out 258.out 185.out 269.out 208.out 039.out 222.out 035.out 169.out 098.out 290.out 080.out 070.out 060.out 207.out 106.out 009.out 167.out 003.out 211.out 158.out 085.out 146.out 188.out 104.out 089.out 086.out 279.out 225.out 174.out 156.out 135.out 257.out 124.out 172.out 058.out 262.out 036.out 042.out 193.out 097.out 271.out 283.out 083.out 199.out 099.out 278.out 053.out 164.out 239.out 241.out 002.out 178.out 141.out 214.out 114.out 074.out 056.out 012.out 041.out 223.out 150.out 123.out 179.out 144.out 032.out 277.out 205.out 136.out 005.out 096.out 198.out 008.out 280.out 050.out 105.out 027.out 187.out 091.out 153.out 088.out 285.out 044.out 201.out 101.out 001.out 266.out 237.out 065.out 231.out 227.out 075.out 131.out 213.out 113.out 013.out 031.out 182.out 272.out 078.out 173.out 259.out 210.out 132.out 037.out 093.out 235.out 127.out 230.out 166.out 130.out 023.out 073.out 030.out 203.out 063.out 055.out 245.out 145.out 159.out 122.out 076.out 022.out 006.out 284.out 299.out 224.out 154.out 276.out 149.out 064.out 218.out 018.out 168.out 007.out 079.out 054.out 059.out 049.out 226.out 163.out 126.out 121.out 026.out 180.out 268.out 253.out 255.out 090.out 221.out 087.out 212.out 112.out 256.out 021.out 247.out 028.out 107.out /var/lib/xfstests/tests/btrfs 2023-07-08T12:40:46.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/ceph/ install 2023-07-08T12:40:46.849 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2023-07-08T12:40:46.858 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2023-07-08T12:40:46.878 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2023-07-08T12:40:46.887 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 003.out 005.out 001.out 004.out 002.out /var/lib/xfstests/tests/ceph 2023-07-08T12:40:46.907 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/cifs/ install 2023-07-08T12:40:46.908 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2023-07-08T12:40:46.917 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2023-07-08T12:40:46.926 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2023-07-08T12:40:46.935 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2023-07-08T12:40:46.944 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/ext4/ install 2023-07-08T12:40:46.945 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2023-07-08T12:40:46.954 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2023-07-08T12:40:47.150 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2023-07-08T12:40:47.159 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 001.out.nodelalloc 001.out.default 001.out.nozero 001.cfg 032.out 039.out 305.out 005.out 008.out 035.out 036.out 308.out 050.out 040.out 306.out 007.out 021.out 046.out 012.out 009.out 033.out 045.out 034.out 056.out 047.out 018.out 023.out 302.out 020.out 002.out 014.out 024.out 013.out 031.out 015.out 304.out 004.out 048.out 038.out 052.out 303.out 029.out 011.out 027.out 042.out 044.out 055.out 030.out 058.out 003.out 017.out 043.out 053.out 016.out 059.out 037.out 301.out 010.out 022.out 006.out 057.out 025.out 026.out 271.out 051.out 041.out 307.out 019.out 028.out 054.out 049.out /var/lib/xfstests/tests/ext4 2023-07-08T12:40:47.366 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/f2fs/ install 2023-07-08T12:40:47.368 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2023-07-08T12:40:47.377 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2023-07-08T12:40:47.389 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2023-07-08T12:40:47.398 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2023-07-08T12:40:47.410 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/generic/ install 2023-07-08T12:40:47.414 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2023-07-08T12:40:47.423 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 /var/lib/xfstests/tests/generic 2023-07-08T12:40:49.492 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2023-07-08T12:40:49.501 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 050.out.xfsquota 035.out.nfs 035.out.default 050.out.nojournal 050.out.default 050.cfg 035.cfg 208.out 433.out 670.out 040.out 104.out 575.out 537.out 683.out 497.out 337.out 702.out 460.out 149.out 088.out 366.out 202.out 137.out 034.out 110.out 389.out 403.out 593.out 571.out 243.out 592.out 282.out 472.out 468.out 628.out 528.out 607.out 428.out 659.out 407.out 228.out 128.out 171.out 028.out 620.out 273.out 474.out 492.out 609.out 707.out 693.out 595.out 506.out 268.out 632.out 532.out 591.out 281.out 471.out 619.out 519.out 013.out 419.out 319.out 177.out 523.out 119.out 019.out 727.out 372.out 686.out 312.out 718.out 143.out 716.out 161.out 516.out 518.out 538.out 655.out 216.out 116.out 016.out 155.out 338.out 238.out 500.out 138.out 189.out 263.out 277.out 181.out 160.out 483.out 274.out 722.out 061.out 239.out 457.out 333.out 258.out 362.out 379.out 691.out 039.out 173.out 498.out 665.out 098.out 711.out 290.out 480.out 706.out 060.out 406.out 306.out 106.out 642.out 447.out 217.out 120.out 191.out 411.out 357.out 481.out 211.out 158.out 014.out 182.out 370.out 188.out 030.out 099.out 585.out 724.out 721.out 624.out 524.out 174.out 156.out 324.out 224.out 313.out 355.out 124.out 382.out 672.out 278.out 063.out 058.out 384.out 296.out 042.out 193.out 271.out 465.out 473.out 335.out 247.out 509.out 568.out 441.out 440.out 475.out 548.out 356.out 341.out 094.out 241.out 600.out 089.out 714.out 614.out 514.out 414.out 314.out 416.out 214.out 381.out 056.out 041.out 057.out 340.out 358.out 572.out 586.out 521.out 240.out 549.out 095.out 633.out 434.out 140.out 708.out 533.out 508.out 408.out 308.out 476.out 087.out 199.out 458.out 233.out 639.out 443.out 540.out 397.out 053.out 279.out 276.out 448.out 164.out 376.out 589.out 587.out 339.out 294.out 352.out 484.out 489.out 139.out 317.out 399.out 220.out 552.out 695.out 017.out 677.out 097.out 175.out 315.out 295.out 580.out 075.out 186.out 452.out 679.out 292.out 252.out 077.out 167.out 688.out 206.out 348.out 394.out 651.out 184.out 466.out 697.out 495.out 725.out 109.out 482.out 625.out 283.out 525.out 488.out 496.out 325.out 551.out 225.out 322.out 636.out 125.out 715.out 192.out 544.out 515.out 650.out 152.out 658.out 563.out 393.out 578.out 015.out 392.out 682.out 684.out 550.out 052.out 652.out 534.out 351.out 583.out 451.out 450.out 463.out 251.out 267.out 176.out 069.out 644.out 065.out 487.out 615.out 395.out 676.out 415.out 344.out 215.out 115.out 444.out 027.out 250.out 391.out 674.out 464.out 705.out 536.out 577.out 439.out 244.out 336.out 144.out 505.out 405.out 342.out 205.out 136.out 598.out 363.out 036.out 198.out 723.out 008.out 390.out 680.out 605.out 305.out 105.out 005.out 367.out 291.out 719.out 323.out 685.out 562.out 570.out 388.out 527.out 237.out 151.out 227.out 163.out 332.out 661.out 582.out 232.out 478.out 584.out 462.out 597.out 412.out 132.out 179.out 037.out 293.out 499.out 638.out 353.out 020.out 660.out 561.out 141.out 076.out 668.out 044.out 648.out 387.out 594.out 438.out 265.out 289.out 574.out 006.out 690.out 284.out 567.out 299.out 456.out 461.out 477.out 307.out 647.out 493.out 107.out 007.out 380.out 526.out 479.out 426.out 150.out 326.out 082.out 226.out 413.out 126.out 025.out 121.out 026.out 021.out 485.out 694.out 347.out 275.out 090.out 361.out 564.out 302.out 432.out 002.out 558.out 162.out 147.out 288.out 197.out 033.out 396.out 051.out 350.out 547.out 047.out 024.out 360.out 543.out 261.out 045.out 692.out 255.out 726.out 378.out 346.out 048.out 062.out 626.out 529.out 231.out 343.out 662.out 425.out 329.out 165.out 334.out 103.out 003.out 129.out 423.out 371.out 234.out 183.out 671.out 645.out 673.out 190.out 486.out 223.out 123.out 262.out 669.out 023.out 459.out 616.out 032.out 266.out 146.out 398.out 728.out 696.out 427.out 590.out 280.out 470.out 455.out 187.out 091.out 046.out 374.out 631.out 328.out 260.out 654.out 359.out 531.out 649.out 301.out 101.out 635.out 623.out 385.out 331.out 556.out 713.out 613.out 513.out 622.out 131.out 213.out 634.out 031.out 565.out 630.out 675.out 272.out 530.out 510.out 627.out 573.out 576.out 430.out 310.out 569.out 454.out 259.out 210.out 449.out 327.out 330.out 067.out 093.out 604.out 287.out 127.out 699.out 166.out 130.out 219.out 073.out 603.out 503.out 303.out 203.out 354.out 157.out 386.out 349.out 055.out 245.out 709.out 522.out 422.out 222.out 159.out 539.out 249.out 122.out 689.out 022.out 681.out 424.out 321.out 066.out 490.out 435.out 375.out 134.out 154.out 566.out 618.out 501.out 418.out 318.out 542.out 218.out 118.out 018.out 377.out 172.out 168.out 079.out 469.out 054.out 059.out 049.out 442.out 657.out 621.out 180.out 270.out 253.out 421.out 712.out 612.out 667.out 512.out 494.out 221.out 212.out 112.out 256.out 012.out 029.out 195.out 554.out 148.out 185.out 596.out 720.out 248.out 196.out 520.out 242.out 420.out 285.out 320.out 629.out 230.out 602.out 502.out 402.out 102.out 717.out 617.out 517.out 429.out 417.out 365.out 092.out 117.out 664.out 072.out 068.out 229.out 038.out 084.out 369.out 611.out 511.out 311.out 368.out 637.out 111.out 011.out 257.out 194.out 170.out 606.out 133.out 286.out 043.out 437.out 545.out 559.out 710.out 610.out 687.out 410.out 701.out 601.out 401.out 201.out 010.out 001.out 096.out 114.out 113.out 074.out 581.out 145.out 641.out 491.out 081.out 071.out 083.out 653.out 700.out 064.out 400.out 300.out 200.out 100.out 646.out 541.out 445.out 269.out 546.out 656.out 698.out 553.out 178.out 298.out 446.out 080.out 070.out 608.out 507.out 409.out 309.out 209.out 108.out 009.out 643.out 004.out 557.out 453.out 666.out 236.out 078.out 364.out 555.out 383.out 085.out 704.out 588.out 504.out 404.out 304.out 204.out 535.out 345.out 246.out 467.out 086.out 254.out 207.out 142.out 235.out 436.out 703.out 135.out 264.out 431.out 678.out 373.out 316.out 169.out 640.out 579.out 297.out 153.out 560.out 663.out 599.out /var/lib/xfstests/tests/generic 2023-07-08T12:40:51.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/nfs/ install 2023-07-08T12:40:51.589 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2023-07-08T12:40:51.598 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2023-07-08T12:40:51.607 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2023-07-08T12:40:51.616 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2023-07-08T12:40:51.625 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/ocfs2/ install 2023-07-08T12:40:51.626 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2023-07-08T12:40:51.635 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2023-07-08T12:40:51.644 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2023-07-08T12:40:51.653 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2023-07-08T12:40:51.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/overlay/ install 2023-07-08T12:40:51.664 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2023-07-08T12:40:51.673 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2023-07-08T12:40:51.951 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2023-07-08T12:40:51.960 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 019.out 034.out 018.out 024.out 077.out 069.out 020.out 102.out 014.out 013.out 031.out 015.out 117.out 017.out 072.out 068.out 063.out 048.out 062.out 038.out 026.out 042.out 052.out 110.out 067.out 029.out 111.out 055.out 112.out 073.out 078.out 030.out 058.out 003.out 045.out 023.out 043.out 053.out 109.out 036.out 116.out 016.out 065.out 010.out 001.out 022.out 106.out 007.out 006.out 115.out 057.out 113.out 114.out 074.out 046.out 071.out 061.out 025.out 051.out 011.out 041.out 047.out 100.out 064.out 107.out 104.out 028.out 076.out 054.out 059.out 008.out 049.out 032.out 039.out 021.out 005.out 002.out 035.out 108.out 070.out 060.out 050.out 027.out 105.out 009.out 004.out 037.out 044.out 012.out 101.out 103.out 033.out 066.out 040.out 056.out 075.out /var/lib/xfstests/tests/overlay 2023-07-08T12:40:52.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/perf/ install 2023-07-08T12:40:52.238 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2023-07-08T12:40:52.247 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2023-07-08T12:40:52.256 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2023-07-08T12:40:52.265 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2023-07-08T12:40:52.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/selftest/ install 2023-07-08T12:40:52.276 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2023-07-08T12:40:52.285 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2023-07-08T12:40:52.308 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2023-07-08T12:40:52.317 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 006.out 003.out 005.out 001.out 004.out 002.out /var/lib/xfstests/tests/selftest 2023-07-08T12:40:52.340 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/shared/ install 2023-07-08T12:40:52.341 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2023-07-08T12:40:52.350 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2023-07-08T12:40:52.365 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2023-07-08T12:40:52.374 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 032.out 298.out 002.out /var/lib/xfstests/tests/shared 2023-07-08T12:40:52.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/tmpfs/ install 2023-07-08T12:40:52.390 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2023-07-08T12:40:52.399 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2023-07-08T12:40:52.408 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2023-07-08T12:40:52.417 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2023-07-08T12:40:52.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/udf/ install 2023-07-08T12:40:52.427 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2023-07-08T12:40:52.436 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2023-07-08T12:40:52.445 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2023-07-08T12:40:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2023-07-08T12:40:52.463 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.HvAMMOKiw7xfstests-dev/tests/xfs/ install 2023-07-08T12:40:52.467 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2023-07-08T12:40:52.476 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2023-07-08T12:40:54.418 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2023-07-08T12:40:54.427 INFO:teuthology.orchestra.run.smithi012.stdout:../../install-sh -o root -g root -m 644 096.out.internal 033.out.crc 096.out.external 033.out.default 071.out.64 216.out.classic 216.out.64mblog 071.out.32 216.cfg 033.cfg 040.out 304.out 349.out 457.out 537.out 295.out 497.out 337.out 385.out 082.out 270.out 460.out 149.out 759.out 502.out 088.out 366.out 202.out 137.out 746.out 110.out 468.out 018.out 593.out 571.out 567.out 592.out 282.out 267.out 728.out 178.out 799.out 528.out 769.out 428.out 507.out 407.out 228.out 128.out 171.out 183.out 247.out 273.out 474.out 109.out 595.out 506.out 732.out 268.out 035.out 113.out 532.out 591.out 281.out 471.out 723.out 519.out 319.out 467.out 523.out 119.out 775.out 727.out 372.out 503.out 526.out 329.out 165.out 229.out 103.out 129.out 371.out 234.out 738.out 078.out 718.out 143.out 716.out 161.out 516.out 416.out 116.out 043.out 155.out 338.out 067.out 238.out 500.out 138.out 189.out 263.out 277.out 181.out 160.out 483.out 274.out 722.out 061.out 239.out 047.out 772.out 258.out 379.out 322.out 039.out 173.out 222.out 742.out 098.out 426.out 290.out 480.out 060.out 406.out 306.out 141.out 106.out 006.out 167.out 191.out 017.out 542.out 211.out 783.out 485.out 442.out 099.out 177.out 174.out 156.out 771.out 324.out 224.out 313.out 124.out 758.out 278.out 058.out 741.out 384.out 773.out 042.out 193.out 392.out 768.out 399.out 473.out 770.out 709.out 755.out 364.out 209.out 441.out 440.out 157.out 740.out 356.out 197.out 341.out 187.out 094.out 241.out 501.out 089.out 540.out 365.out 514.out 414.out 314.out 214.out 381.out 056.out 041.out 340.out 793.out 358.out 572.out 568.out 586.out 240.out 549.out 092.out 439.out 434.out 140.out 708.out 265.out 508.out 408.out 308.out 208.out 737.out 204.out 535.out 104.out 733.out 544.out 059.out 249.out 335.out 142.out 235.out 575.out 253.out 715.out 533.out 264.out 431.out 784.out 433.out 316.out 169.out 579.out 297.out 393.out 333.out 476.out 087.out 199.out 739.out 068.out 233.out 752.out 736.out 443.out 086.out 053.out 749.out 279.out 276.out 711.out 164.out 589.out 587.out 339.out 577.out 294.out 352.out 458.out 498.out 489.out 139.out 774.out 790.out 552.out 781.out 357.out 065.out 186.out 452.out 069.out 252.out 585.out 751.out 724.out 206.out 190.out 794.out 348.out 184.out 435.out 466.out 248.out 725.out 785.out 482.out 750.out 525.out 496.out 325.out 176.out 225.out 125.out 789.out 192.out 296.out 152.out 563.out 246.out 551.out 436.out 792.out 075.out 472.out 578.out 463.out 550.out 052.out 534.out 351.out 430.out 583.out 305.out 451.out 425.out 450.out 744.out 251.out 444.out 487.out 762.out 465.out 395.out 515.out 415.out 389.out 315.out 215.out 115.out 015.out 027.out 250.out 344.out 046.out 051.out 536.out 254.out 030.out 244.out 336.out 028.out 150.out 144.out 505.out 342.out 136.out 005.out 036.out 198.out 008.out 390.out 405.out 588.out 205.out 105.out 367.out 291.out 437.out 719.out 323.out 363.out 562.out 382.out 570.out 388.out 527.out 158.out 766.out 019.out 237.out 151.out 760.out 227.out 163.out 332.out 796.out 232.out 478.out 584.out 462.out 220.out 412.out 132.out 179.out 037.out 293.out 492.out 499.out 112.out 748.out 020.out 786.out 063.out 561.out 076.out 044.out 014.out 362.out 387.out 594.out 438.out 721.out 289.out 574.out 284.out 548.out 299.out 419.out 521.out 581.out 461.out 477.out 307.out 726.out 107.out 007.out 448.out 262.out 380.out 093.out 479.out 547.out 326.out 226.out 345.out 126.out 025.out 121.out 447.out 761.out 580.out 013.out 347.out 275.out 090.out 361.out 564.out 788.out 066.out 455.out 432.out 747.out 558.out 162.out 147.out 288.out 050.out 396.out 576.out 350.out 354.out 298.out 301.out 360.out 543.out 261.out 734.out 045.out 292.out 255.out 729.out 378.out 346.out 048.out 062.out 529.out 231.out 343.out 486.out 223.out 123.out 023.out 459.out 032.out 798.out 146.out 398.out 168.out 427.out 590.out 280.out 470.out 484.out 491.out 767.out 016.out 488.out 731.out 782.out 374.out 285.out 127.out 328.out 260.out 359.out 531.out 376.out 101.out 266.out 331.out 791.out 743.out 556.out 713.out 024.out 513.out 131.out 213.out 730.out 095.out 031.out 565.out 182.out 272.out 530.out 754.out 573.out 454.out 259.out 449.out 327.out 330.out 777.out 493.out 010.out 287.out 230.out 166.out 130.out 219.out 073.out 370.out 135.out 303.out 203.out 003.out 355.out 386.out 560.out 055.out 245.out 243.out 522.out 787.out 745.out 422.out 394.out 159.out 539.out 524.out 122.out 022.out 424.out 779.out 490.out 795.out 353.out 423.out 134.out 154.out 764.out 271.out 518.out 418.out 318.out 077.out 218.out 118.out 757.out 377.out 172.out 034.out 079.out 469.out 054.out 049.out 765.out 021.out 002.out 456.out 180.out 421.out 712.out 403.out 321.out 057.out 512.out 494.out 221.out 312.out 212.out 538.out 256.out 012.out 195.out 283.out 554.out 397.out 185.out 373.out 720.out 481.out 196.out 520.out 242.out 026.out 420.out 188.out 320.out 555.out 120.out 153.out 402.out 302.out 102.out 717.out 517.out 429.out 417.out 317.out 217.out 117.out 475.out 072.out 778.out 097.out 038.out 084.out 369.out 511.out 148.out 411.out 029.out 311.out 368.out 111.out 011.out 257.out 194.out 170.out 133.out 286.out 756.out 175.out 545.out 714.out 559.out 710.out 510.out 413.out 410.out 310.out 210.out 401.out 201.out 001.out 495.out 114.out 753.out 074.out 145.out 797.out 464.out 091.out 081.out 083.out 800.out 064.out 400.out 300.out 200.out 100.out 375.out 541.out 445.out 269.out 546.out 780.out 553.out 763.out 446.out 080.out 070.out 509.out 409.out 309.out 207.out 108.out 009.out 391.out 004.out 557.out 453.out 236.out 566.out 582.out 735.out 383.out 085.out 334.out 504.out 404.out 776.out /var/lib/xfstests/tests/xfs 2023-07-08T12:40:56.408 INFO:teuthology.orchestra.run.smithi012.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2023-07-08T12:40:56.417 INFO:teuthology.orchestra.run.smithi012.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2023-07-08T12:40:56.426 INFO:teuthology.orchestra.run.smithi012.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2023-07-08T12:40:56.435 INFO:teuthology.orchestra.run.smithi012.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2023-07-08T12:40:56.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-08T12:40:56.447 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec ls tests/generic) 2023-07-08T12:40:56.505 INFO:teuthology.orchestra.run.smithi012.stdout:001 2023-07-08T12:40:56.505 INFO:teuthology.orchestra.run.smithi012.stdout:001.out 2023-07-08T12:40:56.505 INFO:teuthology.orchestra.run.smithi012.stdout:002 2023-07-08T12:40:56.505 INFO:teuthology.orchestra.run.smithi012.stdout:002.out 2023-07-08T12:40:56.505 INFO:teuthology.orchestra.run.smithi012.stdout:003 2023-07-08T12:40:56.506 INFO:teuthology.orchestra.run.smithi012.stdout:003.out 2023-07-08T12:40:56.506 INFO:teuthology.orchestra.run.smithi012.stdout:004 2023-07-08T12:40:56.506 INFO:teuthology.orchestra.run.smithi012.stdout:004.out 2023-07-08T12:40:56.506 INFO:teuthology.orchestra.run.smithi012.stdout:005 2023-07-08T12:40:56.506 INFO:teuthology.orchestra.run.smithi012.stdout:005.out 2023-07-08T12:40:56.506 INFO:teuthology.orchestra.run.smithi012.stdout:006 2023-07-08T12:40:56.506 INFO:teuthology.orchestra.run.smithi012.stdout:006.out 2023-07-08T12:40:56.506 INFO:teuthology.orchestra.run.smithi012.stdout:007 2023-07-08T12:40:56.507 INFO:teuthology.orchestra.run.smithi012.stdout:007.out 2023-07-08T12:40:56.507 INFO:teuthology.orchestra.run.smithi012.stdout:008 2023-07-08T12:40:56.507 INFO:teuthology.orchestra.run.smithi012.stdout:008.out 2023-07-08T12:40:56.507 INFO:teuthology.orchestra.run.smithi012.stdout:009 2023-07-08T12:40:56.507 INFO:teuthology.orchestra.run.smithi012.stdout:009.out 2023-07-08T12:40:56.507 INFO:teuthology.orchestra.run.smithi012.stdout:010 2023-07-08T12:40:56.507 INFO:teuthology.orchestra.run.smithi012.stdout:010.out 2023-07-08T12:40:56.507 INFO:teuthology.orchestra.run.smithi012.stdout:011 2023-07-08T12:40:56.508 INFO:teuthology.orchestra.run.smithi012.stdout:011.out 2023-07-08T12:40:56.508 INFO:teuthology.orchestra.run.smithi012.stdout:012 2023-07-08T12:40:56.508 INFO:teuthology.orchestra.run.smithi012.stdout:012.out 2023-07-08T12:40:56.508 INFO:teuthology.orchestra.run.smithi012.stdout:013 2023-07-08T12:40:56.508 INFO:teuthology.orchestra.run.smithi012.stdout:013.out 2023-07-08T12:40:56.508 INFO:teuthology.orchestra.run.smithi012.stdout:014 2023-07-08T12:40:56.508 INFO:teuthology.orchestra.run.smithi012.stdout:014.out 2023-07-08T12:40:56.509 INFO:teuthology.orchestra.run.smithi012.stdout:015 2023-07-08T12:40:56.509 INFO:teuthology.orchestra.run.smithi012.stdout:015.out 2023-07-08T12:40:56.509 INFO:teuthology.orchestra.run.smithi012.stdout:016 2023-07-08T12:40:56.509 INFO:teuthology.orchestra.run.smithi012.stdout:016.out 2023-07-08T12:40:56.509 INFO:teuthology.orchestra.run.smithi012.stdout:017 2023-07-08T12:40:56.509 INFO:teuthology.orchestra.run.smithi012.stdout:017.out 2023-07-08T12:40:56.509 INFO:teuthology.orchestra.run.smithi012.stdout:018 2023-07-08T12:40:56.509 INFO:teuthology.orchestra.run.smithi012.stdout:018.out 2023-07-08T12:40:56.510 INFO:teuthology.orchestra.run.smithi012.stdout:019 2023-07-08T12:40:56.510 INFO:teuthology.orchestra.run.smithi012.stdout:019.out 2023-07-08T12:40:56.510 INFO:teuthology.orchestra.run.smithi012.stdout:020 2023-07-08T12:40:56.510 INFO:teuthology.orchestra.run.smithi012.stdout:020.out 2023-07-08T12:40:56.510 INFO:teuthology.orchestra.run.smithi012.stdout:021 2023-07-08T12:40:56.510 INFO:teuthology.orchestra.run.smithi012.stdout:021.out 2023-07-08T12:40:56.510 INFO:teuthology.orchestra.run.smithi012.stdout:022 2023-07-08T12:40:56.510 INFO:teuthology.orchestra.run.smithi012.stdout:022.out 2023-07-08T12:40:56.511 INFO:teuthology.orchestra.run.smithi012.stdout:023 2023-07-08T12:40:56.511 INFO:teuthology.orchestra.run.smithi012.stdout:023.out 2023-07-08T12:40:56.511 INFO:teuthology.orchestra.run.smithi012.stdout:024 2023-07-08T12:40:56.511 INFO:teuthology.orchestra.run.smithi012.stdout:024.out 2023-07-08T12:40:56.511 INFO:teuthology.orchestra.run.smithi012.stdout:025 2023-07-08T12:40:56.511 INFO:teuthology.orchestra.run.smithi012.stdout:025.out 2023-07-08T12:40:56.511 INFO:teuthology.orchestra.run.smithi012.stdout:026 2023-07-08T12:40:56.511 INFO:teuthology.orchestra.run.smithi012.stdout:026.out 2023-07-08T12:40:56.512 INFO:teuthology.orchestra.run.smithi012.stdout:027 2023-07-08T12:40:56.512 INFO:teuthology.orchestra.run.smithi012.stdout:027.out 2023-07-08T12:40:56.512 INFO:teuthology.orchestra.run.smithi012.stdout:028 2023-07-08T12:40:56.512 INFO:teuthology.orchestra.run.smithi012.stdout:028.out 2023-07-08T12:40:56.512 INFO:teuthology.orchestra.run.smithi012.stdout:029 2023-07-08T12:40:56.512 INFO:teuthology.orchestra.run.smithi012.stdout:029.out 2023-07-08T12:40:56.512 INFO:teuthology.orchestra.run.smithi012.stdout:030 2023-07-08T12:40:56.512 INFO:teuthology.orchestra.run.smithi012.stdout:030.out 2023-07-08T12:40:56.513 INFO:teuthology.orchestra.run.smithi012.stdout:031 2023-07-08T12:40:56.513 INFO:teuthology.orchestra.run.smithi012.stdout:031.out 2023-07-08T12:40:56.513 INFO:teuthology.orchestra.run.smithi012.stdout:032 2023-07-08T12:40:56.513 INFO:teuthology.orchestra.run.smithi012.stdout:032.out 2023-07-08T12:40:56.513 INFO:teuthology.orchestra.run.smithi012.stdout:033 2023-07-08T12:40:56.513 INFO:teuthology.orchestra.run.smithi012.stdout:033.out 2023-07-08T12:40:56.513 INFO:teuthology.orchestra.run.smithi012.stdout:034 2023-07-08T12:40:56.513 INFO:teuthology.orchestra.run.smithi012.stdout:034.out 2023-07-08T12:40:56.514 INFO:teuthology.orchestra.run.smithi012.stdout:035 2023-07-08T12:40:56.514 INFO:teuthology.orchestra.run.smithi012.stdout:035.cfg 2023-07-08T12:40:56.514 INFO:teuthology.orchestra.run.smithi012.stdout:035.out.default 2023-07-08T12:40:56.514 INFO:teuthology.orchestra.run.smithi012.stdout:035.out.nfs 2023-07-08T12:40:56.514 INFO:teuthology.orchestra.run.smithi012.stdout:036 2023-07-08T12:40:56.514 INFO:teuthology.orchestra.run.smithi012.stdout:036.out 2023-07-08T12:40:56.514 INFO:teuthology.orchestra.run.smithi012.stdout:037 2023-07-08T12:40:56.514 INFO:teuthology.orchestra.run.smithi012.stdout:037.out 2023-07-08T12:40:56.515 INFO:teuthology.orchestra.run.smithi012.stdout:038 2023-07-08T12:40:56.515 INFO:teuthology.orchestra.run.smithi012.stdout:038.out 2023-07-08T12:40:56.515 INFO:teuthology.orchestra.run.smithi012.stdout:039 2023-07-08T12:40:56.515 INFO:teuthology.orchestra.run.smithi012.stdout:039.out 2023-07-08T12:40:56.515 INFO:teuthology.orchestra.run.smithi012.stdout:040 2023-07-08T12:40:56.515 INFO:teuthology.orchestra.run.smithi012.stdout:040.out 2023-07-08T12:40:56.515 INFO:teuthology.orchestra.run.smithi012.stdout:041 2023-07-08T12:40:56.515 INFO:teuthology.orchestra.run.smithi012.stdout:041.out 2023-07-08T12:40:56.516 INFO:teuthology.orchestra.run.smithi012.stdout:042 2023-07-08T12:40:56.516 INFO:teuthology.orchestra.run.smithi012.stdout:042.out 2023-07-08T12:40:56.516 INFO:teuthology.orchestra.run.smithi012.stdout:043 2023-07-08T12:40:56.516 INFO:teuthology.orchestra.run.smithi012.stdout:043.out 2023-07-08T12:40:56.516 INFO:teuthology.orchestra.run.smithi012.stdout:044 2023-07-08T12:40:56.516 INFO:teuthology.orchestra.run.smithi012.stdout:044.out 2023-07-08T12:40:56.516 INFO:teuthology.orchestra.run.smithi012.stdout:045 2023-07-08T12:40:56.517 INFO:teuthology.orchestra.run.smithi012.stdout:045.out 2023-07-08T12:40:56.517 INFO:teuthology.orchestra.run.smithi012.stdout:046 2023-07-08T12:40:56.517 INFO:teuthology.orchestra.run.smithi012.stdout:046.out 2023-07-08T12:40:56.517 INFO:teuthology.orchestra.run.smithi012.stdout:047 2023-07-08T12:40:56.517 INFO:teuthology.orchestra.run.smithi012.stdout:047.out 2023-07-08T12:40:56.517 INFO:teuthology.orchestra.run.smithi012.stdout:048 2023-07-08T12:40:56.517 INFO:teuthology.orchestra.run.smithi012.stdout:048.out 2023-07-08T12:40:56.517 INFO:teuthology.orchestra.run.smithi012.stdout:049 2023-07-08T12:40:56.518 INFO:teuthology.orchestra.run.smithi012.stdout:049.out 2023-07-08T12:40:56.518 INFO:teuthology.orchestra.run.smithi012.stdout:050 2023-07-08T12:40:56.518 INFO:teuthology.orchestra.run.smithi012.stdout:050.cfg 2023-07-08T12:40:56.518 INFO:teuthology.orchestra.run.smithi012.stdout:050.out.default 2023-07-08T12:40:56.518 INFO:teuthology.orchestra.run.smithi012.stdout:050.out.nojournal 2023-07-08T12:40:56.518 INFO:teuthology.orchestra.run.smithi012.stdout:050.out.xfsquota 2023-07-08T12:40:56.518 INFO:teuthology.orchestra.run.smithi012.stdout:051 2023-07-08T12:40:56.518 INFO:teuthology.orchestra.run.smithi012.stdout:051.out 2023-07-08T12:40:56.519 INFO:teuthology.orchestra.run.smithi012.stdout:052 2023-07-08T12:40:56.519 INFO:teuthology.orchestra.run.smithi012.stdout:052.out 2023-07-08T12:40:56.519 INFO:teuthology.orchestra.run.smithi012.stdout:053 2023-07-08T12:40:56.519 INFO:teuthology.orchestra.run.smithi012.stdout:053.out 2023-07-08T12:40:56.519 INFO:teuthology.orchestra.run.smithi012.stdout:054 2023-07-08T12:40:56.519 INFO:teuthology.orchestra.run.smithi012.stdout:054.out 2023-07-08T12:40:56.519 INFO:teuthology.orchestra.run.smithi012.stdout:055 2023-07-08T12:40:56.519 INFO:teuthology.orchestra.run.smithi012.stdout:055.out 2023-07-08T12:40:56.520 INFO:teuthology.orchestra.run.smithi012.stdout:056 2023-07-08T12:40:56.520 INFO:teuthology.orchestra.run.smithi012.stdout:056.out 2023-07-08T12:40:56.520 INFO:teuthology.orchestra.run.smithi012.stdout:057 2023-07-08T12:40:56.520 INFO:teuthology.orchestra.run.smithi012.stdout:057.out 2023-07-08T12:40:56.520 INFO:teuthology.orchestra.run.smithi012.stdout:058 2023-07-08T12:40:56.520 INFO:teuthology.orchestra.run.smithi012.stdout:058.out 2023-07-08T12:40:56.520 INFO:teuthology.orchestra.run.smithi012.stdout:059 2023-07-08T12:40:56.520 INFO:teuthology.orchestra.run.smithi012.stdout:059.out 2023-07-08T12:40:56.521 INFO:teuthology.orchestra.run.smithi012.stdout:060 2023-07-08T12:40:56.521 INFO:teuthology.orchestra.run.smithi012.stdout:060.out 2023-07-08T12:40:56.521 INFO:teuthology.orchestra.run.smithi012.stdout:061 2023-07-08T12:40:56.521 INFO:teuthology.orchestra.run.smithi012.stdout:061.out 2023-07-08T12:40:56.521 INFO:teuthology.orchestra.run.smithi012.stdout:062 2023-07-08T12:40:56.521 INFO:teuthology.orchestra.run.smithi012.stdout:062.out 2023-07-08T12:40:56.521 INFO:teuthology.orchestra.run.smithi012.stdout:063 2023-07-08T12:40:56.521 INFO:teuthology.orchestra.run.smithi012.stdout:063.out 2023-07-08T12:40:56.522 INFO:teuthology.orchestra.run.smithi012.stdout:064 2023-07-08T12:40:56.522 INFO:teuthology.orchestra.run.smithi012.stdout:064.out 2023-07-08T12:40:56.522 INFO:teuthology.orchestra.run.smithi012.stdout:065 2023-07-08T12:40:56.522 INFO:teuthology.orchestra.run.smithi012.stdout:065.out 2023-07-08T12:40:56.522 INFO:teuthology.orchestra.run.smithi012.stdout:066 2023-07-08T12:40:56.522 INFO:teuthology.orchestra.run.smithi012.stdout:066.out 2023-07-08T12:40:56.522 INFO:teuthology.orchestra.run.smithi012.stdout:067 2023-07-08T12:40:56.522 INFO:teuthology.orchestra.run.smithi012.stdout:067.out 2023-07-08T12:40:56.523 INFO:teuthology.orchestra.run.smithi012.stdout:068 2023-07-08T12:40:56.523 INFO:teuthology.orchestra.run.smithi012.stdout:068.out 2023-07-08T12:40:56.523 INFO:teuthology.orchestra.run.smithi012.stdout:069 2023-07-08T12:40:56.523 INFO:teuthology.orchestra.run.smithi012.stdout:069.out 2023-07-08T12:40:56.523 INFO:teuthology.orchestra.run.smithi012.stdout:070 2023-07-08T12:40:56.523 INFO:teuthology.orchestra.run.smithi012.stdout:070.out 2023-07-08T12:40:56.523 INFO:teuthology.orchestra.run.smithi012.stdout:071 2023-07-08T12:40:56.523 INFO:teuthology.orchestra.run.smithi012.stdout:071.out 2023-07-08T12:40:56.524 INFO:teuthology.orchestra.run.smithi012.stdout:072 2023-07-08T12:40:56.524 INFO:teuthology.orchestra.run.smithi012.stdout:072.out 2023-07-08T12:40:56.524 INFO:teuthology.orchestra.run.smithi012.stdout:073 2023-07-08T12:40:56.524 INFO:teuthology.orchestra.run.smithi012.stdout:073.out 2023-07-08T12:40:56.524 INFO:teuthology.orchestra.run.smithi012.stdout:074 2023-07-08T12:40:56.524 INFO:teuthology.orchestra.run.smithi012.stdout:074.out 2023-07-08T12:40:56.524 INFO:teuthology.orchestra.run.smithi012.stdout:075 2023-07-08T12:40:56.524 INFO:teuthology.orchestra.run.smithi012.stdout:075.out 2023-07-08T12:40:56.525 INFO:teuthology.orchestra.run.smithi012.stdout:076 2023-07-08T12:40:56.525 INFO:teuthology.orchestra.run.smithi012.stdout:076.out 2023-07-08T12:40:56.525 INFO:teuthology.orchestra.run.smithi012.stdout:077 2023-07-08T12:40:56.525 INFO:teuthology.orchestra.run.smithi012.stdout:077.out 2023-07-08T12:40:56.525 INFO:teuthology.orchestra.run.smithi012.stdout:078 2023-07-08T12:40:56.525 INFO:teuthology.orchestra.run.smithi012.stdout:078.out 2023-07-08T12:40:56.525 INFO:teuthology.orchestra.run.smithi012.stdout:079 2023-07-08T12:40:56.525 INFO:teuthology.orchestra.run.smithi012.stdout:079.out 2023-07-08T12:40:56.526 INFO:teuthology.orchestra.run.smithi012.stdout:080 2023-07-08T12:40:56.526 INFO:teuthology.orchestra.run.smithi012.stdout:080.out 2023-07-08T12:40:56.526 INFO:teuthology.orchestra.run.smithi012.stdout:081 2023-07-08T12:40:56.526 INFO:teuthology.orchestra.run.smithi012.stdout:081.out 2023-07-08T12:40:56.526 INFO:teuthology.orchestra.run.smithi012.stdout:082 2023-07-08T12:40:56.526 INFO:teuthology.orchestra.run.smithi012.stdout:082.out 2023-07-08T12:40:56.526 INFO:teuthology.orchestra.run.smithi012.stdout:083 2023-07-08T12:40:56.526 INFO:teuthology.orchestra.run.smithi012.stdout:083.out 2023-07-08T12:40:56.527 INFO:teuthology.orchestra.run.smithi012.stdout:084 2023-07-08T12:40:56.527 INFO:teuthology.orchestra.run.smithi012.stdout:084.out 2023-07-08T12:40:56.527 INFO:teuthology.orchestra.run.smithi012.stdout:085 2023-07-08T12:40:56.527 INFO:teuthology.orchestra.run.smithi012.stdout:085.out 2023-07-08T12:40:56.527 INFO:teuthology.orchestra.run.smithi012.stdout:086 2023-07-08T12:40:56.527 INFO:teuthology.orchestra.run.smithi012.stdout:086.out 2023-07-08T12:40:56.527 INFO:teuthology.orchestra.run.smithi012.stdout:087 2023-07-08T12:40:56.527 INFO:teuthology.orchestra.run.smithi012.stdout:087.out 2023-07-08T12:40:56.528 INFO:teuthology.orchestra.run.smithi012.stdout:088 2023-07-08T12:40:56.528 INFO:teuthology.orchestra.run.smithi012.stdout:088.out 2023-07-08T12:40:56.528 INFO:teuthology.orchestra.run.smithi012.stdout:089 2023-07-08T12:40:56.528 INFO:teuthology.orchestra.run.smithi012.stdout:089.out 2023-07-08T12:40:56.528 INFO:teuthology.orchestra.run.smithi012.stdout:090 2023-07-08T12:40:56.528 INFO:teuthology.orchestra.run.smithi012.stdout:090.out 2023-07-08T12:40:56.528 INFO:teuthology.orchestra.run.smithi012.stdout:091 2023-07-08T12:40:56.528 INFO:teuthology.orchestra.run.smithi012.stdout:091.out 2023-07-08T12:40:56.529 INFO:teuthology.orchestra.run.smithi012.stdout:092 2023-07-08T12:40:56.529 INFO:teuthology.orchestra.run.smithi012.stdout:092.out 2023-07-08T12:40:56.529 INFO:teuthology.orchestra.run.smithi012.stdout:093 2023-07-08T12:40:56.529 INFO:teuthology.orchestra.run.smithi012.stdout:093.out 2023-07-08T12:40:56.529 INFO:teuthology.orchestra.run.smithi012.stdout:094 2023-07-08T12:40:56.529 INFO:teuthology.orchestra.run.smithi012.stdout:094.out 2023-07-08T12:40:56.529 INFO:teuthology.orchestra.run.smithi012.stdout:095 2023-07-08T12:40:56.529 INFO:teuthology.orchestra.run.smithi012.stdout:095.out 2023-07-08T12:40:56.530 INFO:teuthology.orchestra.run.smithi012.stdout:096 2023-07-08T12:40:56.530 INFO:teuthology.orchestra.run.smithi012.stdout:096.out 2023-07-08T12:40:56.530 INFO:teuthology.orchestra.run.smithi012.stdout:097 2023-07-08T12:40:56.530 INFO:teuthology.orchestra.run.smithi012.stdout:097.out 2023-07-08T12:40:56.530 INFO:teuthology.orchestra.run.smithi012.stdout:098 2023-07-08T12:40:56.530 INFO:teuthology.orchestra.run.smithi012.stdout:098.out 2023-07-08T12:40:56.530 INFO:teuthology.orchestra.run.smithi012.stdout:099 2023-07-08T12:40:56.530 INFO:teuthology.orchestra.run.smithi012.stdout:099.out 2023-07-08T12:40:56.531 INFO:teuthology.orchestra.run.smithi012.stdout:100 2023-07-08T12:40:56.531 INFO:teuthology.orchestra.run.smithi012.stdout:100.out 2023-07-08T12:40:56.531 INFO:teuthology.orchestra.run.smithi012.stdout:101 2023-07-08T12:40:56.531 INFO:teuthology.orchestra.run.smithi012.stdout:101.out 2023-07-08T12:40:56.531 INFO:teuthology.orchestra.run.smithi012.stdout:102 2023-07-08T12:40:56.531 INFO:teuthology.orchestra.run.smithi012.stdout:102.out 2023-07-08T12:40:56.531 INFO:teuthology.orchestra.run.smithi012.stdout:103 2023-07-08T12:40:56.532 INFO:teuthology.orchestra.run.smithi012.stdout:103.out 2023-07-08T12:40:56.532 INFO:teuthology.orchestra.run.smithi012.stdout:104 2023-07-08T12:40:56.532 INFO:teuthology.orchestra.run.smithi012.stdout:104.out 2023-07-08T12:40:56.532 INFO:teuthology.orchestra.run.smithi012.stdout:105 2023-07-08T12:40:56.532 INFO:teuthology.orchestra.run.smithi012.stdout:105.out 2023-07-08T12:40:56.532 INFO:teuthology.orchestra.run.smithi012.stdout:106 2023-07-08T12:40:56.532 INFO:teuthology.orchestra.run.smithi012.stdout:106.out 2023-07-08T12:40:56.532 INFO:teuthology.orchestra.run.smithi012.stdout:107 2023-07-08T12:40:56.533 INFO:teuthology.orchestra.run.smithi012.stdout:107.out 2023-07-08T12:40:56.533 INFO:teuthology.orchestra.run.smithi012.stdout:108 2023-07-08T12:40:56.533 INFO:teuthology.orchestra.run.smithi012.stdout:108.out 2023-07-08T12:40:56.533 INFO:teuthology.orchestra.run.smithi012.stdout:109 2023-07-08T12:40:56.533 INFO:teuthology.orchestra.run.smithi012.stdout:109.out 2023-07-08T12:40:56.533 INFO:teuthology.orchestra.run.smithi012.stdout:110 2023-07-08T12:40:56.533 INFO:teuthology.orchestra.run.smithi012.stdout:110.out 2023-07-08T12:40:56.533 INFO:teuthology.orchestra.run.smithi012.stdout:111 2023-07-08T12:40:56.534 INFO:teuthology.orchestra.run.smithi012.stdout:111.out 2023-07-08T12:40:56.534 INFO:teuthology.orchestra.run.smithi012.stdout:112 2023-07-08T12:40:56.534 INFO:teuthology.orchestra.run.smithi012.stdout:112.out 2023-07-08T12:40:56.534 INFO:teuthology.orchestra.run.smithi012.stdout:113 2023-07-08T12:40:56.534 INFO:teuthology.orchestra.run.smithi012.stdout:113.out 2023-07-08T12:40:56.534 INFO:teuthology.orchestra.run.smithi012.stdout:114 2023-07-08T12:40:56.534 INFO:teuthology.orchestra.run.smithi012.stdout:114.out 2023-07-08T12:40:56.534 INFO:teuthology.orchestra.run.smithi012.stdout:115 2023-07-08T12:40:56.535 INFO:teuthology.orchestra.run.smithi012.stdout:115.out 2023-07-08T12:40:56.535 INFO:teuthology.orchestra.run.smithi012.stdout:116 2023-07-08T12:40:56.535 INFO:teuthology.orchestra.run.smithi012.stdout:116.out 2023-07-08T12:40:56.535 INFO:teuthology.orchestra.run.smithi012.stdout:117 2023-07-08T12:40:56.535 INFO:teuthology.orchestra.run.smithi012.stdout:117.out 2023-07-08T12:40:56.535 INFO:teuthology.orchestra.run.smithi012.stdout:118 2023-07-08T12:40:56.535 INFO:teuthology.orchestra.run.smithi012.stdout:118.out 2023-07-08T12:40:56.535 INFO:teuthology.orchestra.run.smithi012.stdout:119 2023-07-08T12:40:56.536 INFO:teuthology.orchestra.run.smithi012.stdout:119.out 2023-07-08T12:40:56.536 INFO:teuthology.orchestra.run.smithi012.stdout:120 2023-07-08T12:40:56.536 INFO:teuthology.orchestra.run.smithi012.stdout:120.out 2023-07-08T12:40:56.536 INFO:teuthology.orchestra.run.smithi012.stdout:121 2023-07-08T12:40:56.536 INFO:teuthology.orchestra.run.smithi012.stdout:121.out 2023-07-08T12:40:56.536 INFO:teuthology.orchestra.run.smithi012.stdout:122 2023-07-08T12:40:56.536 INFO:teuthology.orchestra.run.smithi012.stdout:122.out 2023-07-08T12:40:56.536 INFO:teuthology.orchestra.run.smithi012.stdout:123 2023-07-08T12:40:56.537 INFO:teuthology.orchestra.run.smithi012.stdout:123.out 2023-07-08T12:40:56.537 INFO:teuthology.orchestra.run.smithi012.stdout:124 2023-07-08T12:40:56.537 INFO:teuthology.orchestra.run.smithi012.stdout:124.out 2023-07-08T12:40:56.537 INFO:teuthology.orchestra.run.smithi012.stdout:125 2023-07-08T12:40:56.537 INFO:teuthology.orchestra.run.smithi012.stdout:125.out 2023-07-08T12:40:56.537 INFO:teuthology.orchestra.run.smithi012.stdout:126 2023-07-08T12:40:56.537 INFO:teuthology.orchestra.run.smithi012.stdout:126.out 2023-07-08T12:40:56.537 INFO:teuthology.orchestra.run.smithi012.stdout:127 2023-07-08T12:40:56.538 INFO:teuthology.orchestra.run.smithi012.stdout:127.out 2023-07-08T12:40:56.538 INFO:teuthology.orchestra.run.smithi012.stdout:128 2023-07-08T12:40:56.538 INFO:teuthology.orchestra.run.smithi012.stdout:128.out 2023-07-08T12:40:56.538 INFO:teuthology.orchestra.run.smithi012.stdout:129 2023-07-08T12:40:56.538 INFO:teuthology.orchestra.run.smithi012.stdout:129.out 2023-07-08T12:40:56.538 INFO:teuthology.orchestra.run.smithi012.stdout:130 2023-07-08T12:40:56.538 INFO:teuthology.orchestra.run.smithi012.stdout:130.out 2023-07-08T12:40:56.539 INFO:teuthology.orchestra.run.smithi012.stdout:131 2023-07-08T12:40:56.539 INFO:teuthology.orchestra.run.smithi012.stdout:131.out 2023-07-08T12:40:56.539 INFO:teuthology.orchestra.run.smithi012.stdout:132 2023-07-08T12:40:56.539 INFO:teuthology.orchestra.run.smithi012.stdout:132.out 2023-07-08T12:40:56.539 INFO:teuthology.orchestra.run.smithi012.stdout:133 2023-07-08T12:40:56.539 INFO:teuthology.orchestra.run.smithi012.stdout:133.out 2023-07-08T12:40:56.539 INFO:teuthology.orchestra.run.smithi012.stdout:134 2023-07-08T12:40:56.539 INFO:teuthology.orchestra.run.smithi012.stdout:134.out 2023-07-08T12:40:56.540 INFO:teuthology.orchestra.run.smithi012.stdout:135 2023-07-08T12:40:56.540 INFO:teuthology.orchestra.run.smithi012.stdout:135.out 2023-07-08T12:40:56.540 INFO:teuthology.orchestra.run.smithi012.stdout:136 2023-07-08T12:40:56.540 INFO:teuthology.orchestra.run.smithi012.stdout:136.out 2023-07-08T12:40:56.540 INFO:teuthology.orchestra.run.smithi012.stdout:137 2023-07-08T12:40:56.540 INFO:teuthology.orchestra.run.smithi012.stdout:137.out 2023-07-08T12:40:56.540 INFO:teuthology.orchestra.run.smithi012.stdout:138 2023-07-08T12:40:56.540 INFO:teuthology.orchestra.run.smithi012.stdout:138.out 2023-07-08T12:40:56.541 INFO:teuthology.orchestra.run.smithi012.stdout:139 2023-07-08T12:40:56.541 INFO:teuthology.orchestra.run.smithi012.stdout:139.out 2023-07-08T12:40:56.541 INFO:teuthology.orchestra.run.smithi012.stdout:140 2023-07-08T12:40:56.541 INFO:teuthology.orchestra.run.smithi012.stdout:140.out 2023-07-08T12:40:56.541 INFO:teuthology.orchestra.run.smithi012.stdout:141 2023-07-08T12:40:56.541 INFO:teuthology.orchestra.run.smithi012.stdout:141.out 2023-07-08T12:40:56.541 INFO:teuthology.orchestra.run.smithi012.stdout:142 2023-07-08T12:40:56.541 INFO:teuthology.orchestra.run.smithi012.stdout:142.out 2023-07-08T12:40:56.542 INFO:teuthology.orchestra.run.smithi012.stdout:143 2023-07-08T12:40:56.542 INFO:teuthology.orchestra.run.smithi012.stdout:143.out 2023-07-08T12:40:56.542 INFO:teuthology.orchestra.run.smithi012.stdout:144 2023-07-08T12:40:56.542 INFO:teuthology.orchestra.run.smithi012.stdout:144.out 2023-07-08T12:40:56.542 INFO:teuthology.orchestra.run.smithi012.stdout:145 2023-07-08T12:40:56.542 INFO:teuthology.orchestra.run.smithi012.stdout:145.out 2023-07-08T12:40:56.542 INFO:teuthology.orchestra.run.smithi012.stdout:146 2023-07-08T12:40:56.542 INFO:teuthology.orchestra.run.smithi012.stdout:146.out 2023-07-08T12:40:56.543 INFO:teuthology.orchestra.run.smithi012.stdout:147 2023-07-08T12:40:56.543 INFO:teuthology.orchestra.run.smithi012.stdout:147.out 2023-07-08T12:40:56.543 INFO:teuthology.orchestra.run.smithi012.stdout:148 2023-07-08T12:40:56.543 INFO:teuthology.orchestra.run.smithi012.stdout:148.out 2023-07-08T12:40:56.543 INFO:teuthology.orchestra.run.smithi012.stdout:149 2023-07-08T12:40:56.543 INFO:teuthology.orchestra.run.smithi012.stdout:149.out 2023-07-08T12:40:56.543 INFO:teuthology.orchestra.run.smithi012.stdout:150 2023-07-08T12:40:56.543 INFO:teuthology.orchestra.run.smithi012.stdout:150.out 2023-07-08T12:40:56.544 INFO:teuthology.orchestra.run.smithi012.stdout:151 2023-07-08T12:40:56.544 INFO:teuthology.orchestra.run.smithi012.stdout:151.out 2023-07-08T12:40:56.544 INFO:teuthology.orchestra.run.smithi012.stdout:152 2023-07-08T12:40:56.544 INFO:teuthology.orchestra.run.smithi012.stdout:152.out 2023-07-08T12:40:56.544 INFO:teuthology.orchestra.run.smithi012.stdout:153 2023-07-08T12:40:56.544 INFO:teuthology.orchestra.run.smithi012.stdout:153.out 2023-07-08T12:40:56.544 INFO:teuthology.orchestra.run.smithi012.stdout:154 2023-07-08T12:40:56.544 INFO:teuthology.orchestra.run.smithi012.stdout:154.out 2023-07-08T12:40:56.545 INFO:teuthology.orchestra.run.smithi012.stdout:155 2023-07-08T12:40:56.545 INFO:teuthology.orchestra.run.smithi012.stdout:155.out 2023-07-08T12:40:56.545 INFO:teuthology.orchestra.run.smithi012.stdout:156 2023-07-08T12:40:56.545 INFO:teuthology.orchestra.run.smithi012.stdout:156.out 2023-07-08T12:40:56.545 INFO:teuthology.orchestra.run.smithi012.stdout:157 2023-07-08T12:40:56.545 INFO:teuthology.orchestra.run.smithi012.stdout:157.out 2023-07-08T12:40:56.545 INFO:teuthology.orchestra.run.smithi012.stdout:158 2023-07-08T12:40:56.545 INFO:teuthology.orchestra.run.smithi012.stdout:158.out 2023-07-08T12:40:56.546 INFO:teuthology.orchestra.run.smithi012.stdout:159 2023-07-08T12:40:56.546 INFO:teuthology.orchestra.run.smithi012.stdout:159.out 2023-07-08T12:40:56.546 INFO:teuthology.orchestra.run.smithi012.stdout:160 2023-07-08T12:40:56.546 INFO:teuthology.orchestra.run.smithi012.stdout:160.out 2023-07-08T12:40:56.546 INFO:teuthology.orchestra.run.smithi012.stdout:161 2023-07-08T12:40:56.546 INFO:teuthology.orchestra.run.smithi012.stdout:161.out 2023-07-08T12:40:56.546 INFO:teuthology.orchestra.run.smithi012.stdout:162 2023-07-08T12:40:56.546 INFO:teuthology.orchestra.run.smithi012.stdout:162.out 2023-07-08T12:40:56.547 INFO:teuthology.orchestra.run.smithi012.stdout:163 2023-07-08T12:40:56.547 INFO:teuthology.orchestra.run.smithi012.stdout:163.out 2023-07-08T12:40:56.547 INFO:teuthology.orchestra.run.smithi012.stdout:164 2023-07-08T12:40:56.547 INFO:teuthology.orchestra.run.smithi012.stdout:164.out 2023-07-08T12:40:56.547 INFO:teuthology.orchestra.run.smithi012.stdout:165 2023-07-08T12:40:56.547 INFO:teuthology.orchestra.run.smithi012.stdout:165.out 2023-07-08T12:40:56.547 INFO:teuthology.orchestra.run.smithi012.stdout:166 2023-07-08T12:40:56.547 INFO:teuthology.orchestra.run.smithi012.stdout:166.out 2023-07-08T12:40:56.548 INFO:teuthology.orchestra.run.smithi012.stdout:167 2023-07-08T12:40:56.548 INFO:teuthology.orchestra.run.smithi012.stdout:167.out 2023-07-08T12:40:56.548 INFO:teuthology.orchestra.run.smithi012.stdout:168 2023-07-08T12:40:56.548 INFO:teuthology.orchestra.run.smithi012.stdout:168.out 2023-07-08T12:40:56.548 INFO:teuthology.orchestra.run.smithi012.stdout:169 2023-07-08T12:40:56.548 INFO:teuthology.orchestra.run.smithi012.stdout:169.out 2023-07-08T12:40:56.548 INFO:teuthology.orchestra.run.smithi012.stdout:170 2023-07-08T12:40:56.548 INFO:teuthology.orchestra.run.smithi012.stdout:170.out 2023-07-08T12:40:56.549 INFO:teuthology.orchestra.run.smithi012.stdout:171 2023-07-08T12:40:56.549 INFO:teuthology.orchestra.run.smithi012.stdout:171.out 2023-07-08T12:40:56.549 INFO:teuthology.orchestra.run.smithi012.stdout:172 2023-07-08T12:40:56.549 INFO:teuthology.orchestra.run.smithi012.stdout:172.out 2023-07-08T12:40:56.549 INFO:teuthology.orchestra.run.smithi012.stdout:173 2023-07-08T12:40:56.549 INFO:teuthology.orchestra.run.smithi012.stdout:173.out 2023-07-08T12:40:56.549 INFO:teuthology.orchestra.run.smithi012.stdout:174 2023-07-08T12:40:56.549 INFO:teuthology.orchestra.run.smithi012.stdout:174.out 2023-07-08T12:40:56.550 INFO:teuthology.orchestra.run.smithi012.stdout:175 2023-07-08T12:40:56.550 INFO:teuthology.orchestra.run.smithi012.stdout:175.out 2023-07-08T12:40:56.550 INFO:teuthology.orchestra.run.smithi012.stdout:176 2023-07-08T12:40:56.550 INFO:teuthology.orchestra.run.smithi012.stdout:176.out 2023-07-08T12:40:56.550 INFO:teuthology.orchestra.run.smithi012.stdout:177 2023-07-08T12:40:56.550 INFO:teuthology.orchestra.run.smithi012.stdout:177.out 2023-07-08T12:40:56.550 INFO:teuthology.orchestra.run.smithi012.stdout:178 2023-07-08T12:40:56.550 INFO:teuthology.orchestra.run.smithi012.stdout:178.out 2023-07-08T12:40:56.551 INFO:teuthology.orchestra.run.smithi012.stdout:179 2023-07-08T12:40:56.551 INFO:teuthology.orchestra.run.smithi012.stdout:179.out 2023-07-08T12:40:56.551 INFO:teuthology.orchestra.run.smithi012.stdout:180 2023-07-08T12:40:56.551 INFO:teuthology.orchestra.run.smithi012.stdout:180.out 2023-07-08T12:40:56.551 INFO:teuthology.orchestra.run.smithi012.stdout:181 2023-07-08T12:40:56.551 INFO:teuthology.orchestra.run.smithi012.stdout:181.out 2023-07-08T12:40:56.551 INFO:teuthology.orchestra.run.smithi012.stdout:182 2023-07-08T12:40:56.551 INFO:teuthology.orchestra.run.smithi012.stdout:182.out 2023-07-08T12:40:56.552 INFO:teuthology.orchestra.run.smithi012.stdout:183 2023-07-08T12:40:56.552 INFO:teuthology.orchestra.run.smithi012.stdout:183.out 2023-07-08T12:40:56.552 INFO:teuthology.orchestra.run.smithi012.stdout:184 2023-07-08T12:40:56.552 INFO:teuthology.orchestra.run.smithi012.stdout:184.out 2023-07-08T12:40:56.552 INFO:teuthology.orchestra.run.smithi012.stdout:185 2023-07-08T12:40:56.552 INFO:teuthology.orchestra.run.smithi012.stdout:185.out 2023-07-08T12:40:56.552 INFO:teuthology.orchestra.run.smithi012.stdout:186 2023-07-08T12:40:56.552 INFO:teuthology.orchestra.run.smithi012.stdout:186.out 2023-07-08T12:40:56.553 INFO:teuthology.orchestra.run.smithi012.stdout:187 2023-07-08T12:40:56.553 INFO:teuthology.orchestra.run.smithi012.stdout:187.out 2023-07-08T12:40:56.553 INFO:teuthology.orchestra.run.smithi012.stdout:188 2023-07-08T12:40:56.553 INFO:teuthology.orchestra.run.smithi012.stdout:188.out 2023-07-08T12:40:56.553 INFO:teuthology.orchestra.run.smithi012.stdout:189 2023-07-08T12:40:56.553 INFO:teuthology.orchestra.run.smithi012.stdout:189.out 2023-07-08T12:40:56.553 INFO:teuthology.orchestra.run.smithi012.stdout:190 2023-07-08T12:40:56.553 INFO:teuthology.orchestra.run.smithi012.stdout:190.out 2023-07-08T12:40:56.554 INFO:teuthology.orchestra.run.smithi012.stdout:191 2023-07-08T12:40:56.554 INFO:teuthology.orchestra.run.smithi012.stdout:191.out 2023-07-08T12:40:56.554 INFO:teuthology.orchestra.run.smithi012.stdout:192 2023-07-08T12:40:56.554 INFO:teuthology.orchestra.run.smithi012.stdout:192.out 2023-07-08T12:40:56.554 INFO:teuthology.orchestra.run.smithi012.stdout:193 2023-07-08T12:40:56.554 INFO:teuthology.orchestra.run.smithi012.stdout:193.out 2023-07-08T12:40:56.554 INFO:teuthology.orchestra.run.smithi012.stdout:194 2023-07-08T12:40:56.554 INFO:teuthology.orchestra.run.smithi012.stdout:194.out 2023-07-08T12:40:56.555 INFO:teuthology.orchestra.run.smithi012.stdout:195 2023-07-08T12:40:56.555 INFO:teuthology.orchestra.run.smithi012.stdout:195.out 2023-07-08T12:40:56.555 INFO:teuthology.orchestra.run.smithi012.stdout:196 2023-07-08T12:40:56.555 INFO:teuthology.orchestra.run.smithi012.stdout:196.out 2023-07-08T12:40:56.555 INFO:teuthology.orchestra.run.smithi012.stdout:197 2023-07-08T12:40:56.555 INFO:teuthology.orchestra.run.smithi012.stdout:197.out 2023-07-08T12:40:56.555 INFO:teuthology.orchestra.run.smithi012.stdout:198 2023-07-08T12:40:56.555 INFO:teuthology.orchestra.run.smithi012.stdout:198.out 2023-07-08T12:40:56.556 INFO:teuthology.orchestra.run.smithi012.stdout:199 2023-07-08T12:40:56.556 INFO:teuthology.orchestra.run.smithi012.stdout:199.out 2023-07-08T12:40:56.556 INFO:teuthology.orchestra.run.smithi012.stdout:200 2023-07-08T12:40:56.556 INFO:teuthology.orchestra.run.smithi012.stdout:200.out 2023-07-08T12:40:56.556 INFO:teuthology.orchestra.run.smithi012.stdout:201 2023-07-08T12:40:56.556 INFO:teuthology.orchestra.run.smithi012.stdout:201.out 2023-07-08T12:40:56.556 INFO:teuthology.orchestra.run.smithi012.stdout:202 2023-07-08T12:40:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:202.out 2023-07-08T12:40:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:203 2023-07-08T12:40:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:203.out 2023-07-08T12:40:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:204 2023-07-08T12:40:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:204.out 2023-07-08T12:40:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:205 2023-07-08T12:40:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:205.out 2023-07-08T12:40:56.557 INFO:teuthology.orchestra.run.smithi012.stdout:206 2023-07-08T12:40:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:206.out 2023-07-08T12:40:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:207 2023-07-08T12:40:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:207.out 2023-07-08T12:40:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:208 2023-07-08T12:40:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:208.out 2023-07-08T12:40:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:209 2023-07-08T12:40:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:209.out 2023-07-08T12:40:56.558 INFO:teuthology.orchestra.run.smithi012.stdout:210 2023-07-08T12:40:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:210.out 2023-07-08T12:40:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:211 2023-07-08T12:40:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:211.out 2023-07-08T12:40:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:212 2023-07-08T12:40:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:212.out 2023-07-08T12:40:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:213 2023-07-08T12:40:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:213.out 2023-07-08T12:40:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:214 2023-07-08T12:40:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:214.out 2023-07-08T12:40:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:215 2023-07-08T12:40:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:215.out 2023-07-08T12:40:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:216 2023-07-08T12:40:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:216.out 2023-07-08T12:40:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:217 2023-07-08T12:40:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:217.out 2023-07-08T12:40:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:218 2023-07-08T12:40:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:218.out 2023-07-08T12:40:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:219 2023-07-08T12:40:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:219.out 2023-07-08T12:40:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:220 2023-07-08T12:40:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:220.out 2023-07-08T12:40:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:221 2023-07-08T12:40:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:221.out 2023-07-08T12:40:56.561 INFO:teuthology.orchestra.run.smithi012.stdout:222 2023-07-08T12:40:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:222.out 2023-07-08T12:40:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:223 2023-07-08T12:40:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:223.out 2023-07-08T12:40:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:224 2023-07-08T12:40:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:224.out 2023-07-08T12:40:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:225 2023-07-08T12:40:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:225.out 2023-07-08T12:40:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:226 2023-07-08T12:40:56.563 INFO:teuthology.orchestra.run.smithi012.stdout:226.out 2023-07-08T12:40:56.563 INFO:teuthology.orchestra.run.smithi012.stdout:227 2023-07-08T12:40:56.563 INFO:teuthology.orchestra.run.smithi012.stdout:227.out 2023-07-08T12:40:56.563 INFO:teuthology.orchestra.run.smithi012.stdout:228 2023-07-08T12:40:56.563 INFO:teuthology.orchestra.run.smithi012.stdout:228.out 2023-07-08T12:40:56.563 INFO:teuthology.orchestra.run.smithi012.stdout:229 2023-07-08T12:40:56.563 INFO:teuthology.orchestra.run.smithi012.stdout:229.out 2023-07-08T12:40:56.563 INFO:teuthology.orchestra.run.smithi012.stdout:230 2023-07-08T12:40:56.564 INFO:teuthology.orchestra.run.smithi012.stdout:230.out 2023-07-08T12:40:56.564 INFO:teuthology.orchestra.run.smithi012.stdout:231 2023-07-08T12:40:56.564 INFO:teuthology.orchestra.run.smithi012.stdout:231.out 2023-07-08T12:40:56.564 INFO:teuthology.orchestra.run.smithi012.stdout:232 2023-07-08T12:40:56.564 INFO:teuthology.orchestra.run.smithi012.stdout:232.out 2023-07-08T12:40:56.564 INFO:teuthology.orchestra.run.smithi012.stdout:233 2023-07-08T12:40:56.564 INFO:teuthology.orchestra.run.smithi012.stdout:233.out 2023-07-08T12:40:56.564 INFO:teuthology.orchestra.run.smithi012.stdout:234 2023-07-08T12:40:56.565 INFO:teuthology.orchestra.run.smithi012.stdout:234.out 2023-07-08T12:40:56.565 INFO:teuthology.orchestra.run.smithi012.stdout:235 2023-07-08T12:40:56.565 INFO:teuthology.orchestra.run.smithi012.stdout:235.out 2023-07-08T12:40:56.565 INFO:teuthology.orchestra.run.smithi012.stdout:236 2023-07-08T12:40:56.565 INFO:teuthology.orchestra.run.smithi012.stdout:236.out 2023-07-08T12:40:56.565 INFO:teuthology.orchestra.run.smithi012.stdout:237 2023-07-08T12:40:56.565 INFO:teuthology.orchestra.run.smithi012.stdout:237.out 2023-07-08T12:40:56.565 INFO:teuthology.orchestra.run.smithi012.stdout:238 2023-07-08T12:40:56.566 INFO:teuthology.orchestra.run.smithi012.stdout:238.out 2023-07-08T12:40:56.566 INFO:teuthology.orchestra.run.smithi012.stdout:239 2023-07-08T12:40:56.566 INFO:teuthology.orchestra.run.smithi012.stdout:239.out 2023-07-08T12:40:56.566 INFO:teuthology.orchestra.run.smithi012.stdout:240 2023-07-08T12:40:56.566 INFO:teuthology.orchestra.run.smithi012.stdout:240.out 2023-07-08T12:40:56.566 INFO:teuthology.orchestra.run.smithi012.stdout:241 2023-07-08T12:40:56.566 INFO:teuthology.orchestra.run.smithi012.stdout:241.out 2023-07-08T12:40:56.566 INFO:teuthology.orchestra.run.smithi012.stdout:242 2023-07-08T12:40:56.567 INFO:teuthology.orchestra.run.smithi012.stdout:242.out 2023-07-08T12:40:56.567 INFO:teuthology.orchestra.run.smithi012.stdout:243 2023-07-08T12:40:56.567 INFO:teuthology.orchestra.run.smithi012.stdout:243.out 2023-07-08T12:40:56.567 INFO:teuthology.orchestra.run.smithi012.stdout:244 2023-07-08T12:40:56.567 INFO:teuthology.orchestra.run.smithi012.stdout:244.out 2023-07-08T12:40:56.567 INFO:teuthology.orchestra.run.smithi012.stdout:245 2023-07-08T12:40:56.567 INFO:teuthology.orchestra.run.smithi012.stdout:245.out 2023-07-08T12:40:56.567 INFO:teuthology.orchestra.run.smithi012.stdout:246 2023-07-08T12:40:56.568 INFO:teuthology.orchestra.run.smithi012.stdout:246.out 2023-07-08T12:40:56.568 INFO:teuthology.orchestra.run.smithi012.stdout:247 2023-07-08T12:40:56.568 INFO:teuthology.orchestra.run.smithi012.stdout:247.out 2023-07-08T12:40:56.568 INFO:teuthology.orchestra.run.smithi012.stdout:248 2023-07-08T12:40:56.568 INFO:teuthology.orchestra.run.smithi012.stdout:248.out 2023-07-08T12:40:56.568 INFO:teuthology.orchestra.run.smithi012.stdout:249 2023-07-08T12:40:56.568 INFO:teuthology.orchestra.run.smithi012.stdout:249.out 2023-07-08T12:40:56.568 INFO:teuthology.orchestra.run.smithi012.stdout:250 2023-07-08T12:40:56.569 INFO:teuthology.orchestra.run.smithi012.stdout:250.out 2023-07-08T12:40:56.569 INFO:teuthology.orchestra.run.smithi012.stdout:251 2023-07-08T12:40:56.569 INFO:teuthology.orchestra.run.smithi012.stdout:251.out 2023-07-08T12:40:56.569 INFO:teuthology.orchestra.run.smithi012.stdout:252 2023-07-08T12:40:56.569 INFO:teuthology.orchestra.run.smithi012.stdout:252.out 2023-07-08T12:40:56.569 INFO:teuthology.orchestra.run.smithi012.stdout:253 2023-07-08T12:40:56.569 INFO:teuthology.orchestra.run.smithi012.stdout:253.out 2023-07-08T12:40:56.569 INFO:teuthology.orchestra.run.smithi012.stdout:254 2023-07-08T12:40:56.570 INFO:teuthology.orchestra.run.smithi012.stdout:254.out 2023-07-08T12:40:56.570 INFO:teuthology.orchestra.run.smithi012.stdout:255 2023-07-08T12:40:56.570 INFO:teuthology.orchestra.run.smithi012.stdout:255.out 2023-07-08T12:40:56.570 INFO:teuthology.orchestra.run.smithi012.stdout:256 2023-07-08T12:40:56.570 INFO:teuthology.orchestra.run.smithi012.stdout:256.out 2023-07-08T12:40:56.570 INFO:teuthology.orchestra.run.smithi012.stdout:257 2023-07-08T12:40:56.570 INFO:teuthology.orchestra.run.smithi012.stdout:257.out 2023-07-08T12:40:56.570 INFO:teuthology.orchestra.run.smithi012.stdout:258 2023-07-08T12:40:56.571 INFO:teuthology.orchestra.run.smithi012.stdout:258.out 2023-07-08T12:40:56.571 INFO:teuthology.orchestra.run.smithi012.stdout:259 2023-07-08T12:40:56.571 INFO:teuthology.orchestra.run.smithi012.stdout:259.out 2023-07-08T12:40:56.571 INFO:teuthology.orchestra.run.smithi012.stdout:260 2023-07-08T12:40:56.571 INFO:teuthology.orchestra.run.smithi012.stdout:260.out 2023-07-08T12:40:56.571 INFO:teuthology.orchestra.run.smithi012.stdout:261 2023-07-08T12:40:56.571 INFO:teuthology.orchestra.run.smithi012.stdout:261.out 2023-07-08T12:40:56.571 INFO:teuthology.orchestra.run.smithi012.stdout:262 2023-07-08T12:40:56.572 INFO:teuthology.orchestra.run.smithi012.stdout:262.out 2023-07-08T12:40:56.572 INFO:teuthology.orchestra.run.smithi012.stdout:263 2023-07-08T12:40:56.572 INFO:teuthology.orchestra.run.smithi012.stdout:263.out 2023-07-08T12:40:56.572 INFO:teuthology.orchestra.run.smithi012.stdout:264 2023-07-08T12:40:56.572 INFO:teuthology.orchestra.run.smithi012.stdout:264.out 2023-07-08T12:40:56.572 INFO:teuthology.orchestra.run.smithi012.stdout:265 2023-07-08T12:40:56.572 INFO:teuthology.orchestra.run.smithi012.stdout:265.out 2023-07-08T12:40:56.572 INFO:teuthology.orchestra.run.smithi012.stdout:266 2023-07-08T12:40:56.573 INFO:teuthology.orchestra.run.smithi012.stdout:266.out 2023-07-08T12:40:56.573 INFO:teuthology.orchestra.run.smithi012.stdout:267 2023-07-08T12:40:56.573 INFO:teuthology.orchestra.run.smithi012.stdout:267.out 2023-07-08T12:40:56.573 INFO:teuthology.orchestra.run.smithi012.stdout:268 2023-07-08T12:40:56.573 INFO:teuthology.orchestra.run.smithi012.stdout:268.out 2023-07-08T12:40:56.573 INFO:teuthology.orchestra.run.smithi012.stdout:269 2023-07-08T12:40:56.573 INFO:teuthology.orchestra.run.smithi012.stdout:269.out 2023-07-08T12:40:56.573 INFO:teuthology.orchestra.run.smithi012.stdout:270 2023-07-08T12:40:56.574 INFO:teuthology.orchestra.run.smithi012.stdout:270.out 2023-07-08T12:40:56.574 INFO:teuthology.orchestra.run.smithi012.stdout:271 2023-07-08T12:40:56.574 INFO:teuthology.orchestra.run.smithi012.stdout:271.out 2023-07-08T12:40:56.574 INFO:teuthology.orchestra.run.smithi012.stdout:272 2023-07-08T12:40:56.574 INFO:teuthology.orchestra.run.smithi012.stdout:272.out 2023-07-08T12:40:56.574 INFO:teuthology.orchestra.run.smithi012.stdout:273 2023-07-08T12:40:56.574 INFO:teuthology.orchestra.run.smithi012.stdout:273.out 2023-07-08T12:40:56.574 INFO:teuthology.orchestra.run.smithi012.stdout:274 2023-07-08T12:40:56.575 INFO:teuthology.orchestra.run.smithi012.stdout:274.out 2023-07-08T12:40:56.575 INFO:teuthology.orchestra.run.smithi012.stdout:275 2023-07-08T12:40:56.575 INFO:teuthology.orchestra.run.smithi012.stdout:275.out 2023-07-08T12:40:56.575 INFO:teuthology.orchestra.run.smithi012.stdout:276 2023-07-08T12:40:56.575 INFO:teuthology.orchestra.run.smithi012.stdout:276.out 2023-07-08T12:40:56.575 INFO:teuthology.orchestra.run.smithi012.stdout:277 2023-07-08T12:40:56.575 INFO:teuthology.orchestra.run.smithi012.stdout:277.out 2023-07-08T12:40:56.575 INFO:teuthology.orchestra.run.smithi012.stdout:278 2023-07-08T12:40:56.576 INFO:teuthology.orchestra.run.smithi012.stdout:278.out 2023-07-08T12:40:56.576 INFO:teuthology.orchestra.run.smithi012.stdout:279 2023-07-08T12:40:56.576 INFO:teuthology.orchestra.run.smithi012.stdout:279.out 2023-07-08T12:40:56.576 INFO:teuthology.orchestra.run.smithi012.stdout:280 2023-07-08T12:40:56.576 INFO:teuthology.orchestra.run.smithi012.stdout:280.out 2023-07-08T12:40:56.576 INFO:teuthology.orchestra.run.smithi012.stdout:281 2023-07-08T12:40:56.576 INFO:teuthology.orchestra.run.smithi012.stdout:281.out 2023-07-08T12:40:56.576 INFO:teuthology.orchestra.run.smithi012.stdout:282 2023-07-08T12:40:56.577 INFO:teuthology.orchestra.run.smithi012.stdout:282.out 2023-07-08T12:40:56.577 INFO:teuthology.orchestra.run.smithi012.stdout:283 2023-07-08T12:40:56.577 INFO:teuthology.orchestra.run.smithi012.stdout:283.out 2023-07-08T12:40:56.577 INFO:teuthology.orchestra.run.smithi012.stdout:284 2023-07-08T12:40:56.577 INFO:teuthology.orchestra.run.smithi012.stdout:284.out 2023-07-08T12:40:56.577 INFO:teuthology.orchestra.run.smithi012.stdout:285 2023-07-08T12:40:56.577 INFO:teuthology.orchestra.run.smithi012.stdout:285.out 2023-07-08T12:40:56.577 INFO:teuthology.orchestra.run.smithi012.stdout:286 2023-07-08T12:40:56.578 INFO:teuthology.orchestra.run.smithi012.stdout:286.out 2023-07-08T12:40:56.578 INFO:teuthology.orchestra.run.smithi012.stdout:287 2023-07-08T12:40:56.578 INFO:teuthology.orchestra.run.smithi012.stdout:287.out 2023-07-08T12:40:56.578 INFO:teuthology.orchestra.run.smithi012.stdout:288 2023-07-08T12:40:56.578 INFO:teuthology.orchestra.run.smithi012.stdout:288.out 2023-07-08T12:40:56.578 INFO:teuthology.orchestra.run.smithi012.stdout:289 2023-07-08T12:40:56.578 INFO:teuthology.orchestra.run.smithi012.stdout:289.out 2023-07-08T12:40:56.578 INFO:teuthology.orchestra.run.smithi012.stdout:290 2023-07-08T12:40:56.579 INFO:teuthology.orchestra.run.smithi012.stdout:290.out 2023-07-08T12:40:56.579 INFO:teuthology.orchestra.run.smithi012.stdout:291 2023-07-08T12:40:56.579 INFO:teuthology.orchestra.run.smithi012.stdout:291.out 2023-07-08T12:40:56.579 INFO:teuthology.orchestra.run.smithi012.stdout:292 2023-07-08T12:40:56.579 INFO:teuthology.orchestra.run.smithi012.stdout:292.out 2023-07-08T12:40:56.579 INFO:teuthology.orchestra.run.smithi012.stdout:293 2023-07-08T12:40:56.579 INFO:teuthology.orchestra.run.smithi012.stdout:293.out 2023-07-08T12:40:56.579 INFO:teuthology.orchestra.run.smithi012.stdout:294 2023-07-08T12:40:56.580 INFO:teuthology.orchestra.run.smithi012.stdout:294.out 2023-07-08T12:40:56.580 INFO:teuthology.orchestra.run.smithi012.stdout:295 2023-07-08T12:40:56.580 INFO:teuthology.orchestra.run.smithi012.stdout:295.out 2023-07-08T12:40:56.580 INFO:teuthology.orchestra.run.smithi012.stdout:296 2023-07-08T12:40:56.580 INFO:teuthology.orchestra.run.smithi012.stdout:296.out 2023-07-08T12:40:56.580 INFO:teuthology.orchestra.run.smithi012.stdout:297 2023-07-08T12:40:56.580 INFO:teuthology.orchestra.run.smithi012.stdout:297.out 2023-07-08T12:40:56.580 INFO:teuthology.orchestra.run.smithi012.stdout:298 2023-07-08T12:40:56.581 INFO:teuthology.orchestra.run.smithi012.stdout:298.out 2023-07-08T12:40:56.581 INFO:teuthology.orchestra.run.smithi012.stdout:299 2023-07-08T12:40:56.581 INFO:teuthology.orchestra.run.smithi012.stdout:299.out 2023-07-08T12:40:56.581 INFO:teuthology.orchestra.run.smithi012.stdout:300 2023-07-08T12:40:56.581 INFO:teuthology.orchestra.run.smithi012.stdout:300.out 2023-07-08T12:40:56.581 INFO:teuthology.orchestra.run.smithi012.stdout:301 2023-07-08T12:40:56.581 INFO:teuthology.orchestra.run.smithi012.stdout:301.out 2023-07-08T12:40:56.581 INFO:teuthology.orchestra.run.smithi012.stdout:302 2023-07-08T12:40:56.582 INFO:teuthology.orchestra.run.smithi012.stdout:302.out 2023-07-08T12:40:56.582 INFO:teuthology.orchestra.run.smithi012.stdout:303 2023-07-08T12:40:56.582 INFO:teuthology.orchestra.run.smithi012.stdout:303.out 2023-07-08T12:40:56.582 INFO:teuthology.orchestra.run.smithi012.stdout:304 2023-07-08T12:40:56.582 INFO:teuthology.orchestra.run.smithi012.stdout:304.out 2023-07-08T12:40:56.582 INFO:teuthology.orchestra.run.smithi012.stdout:305 2023-07-08T12:40:56.582 INFO:teuthology.orchestra.run.smithi012.stdout:305.out 2023-07-08T12:40:56.582 INFO:teuthology.orchestra.run.smithi012.stdout:306 2023-07-08T12:40:56.583 INFO:teuthology.orchestra.run.smithi012.stdout:306.out 2023-07-08T12:40:56.583 INFO:teuthology.orchestra.run.smithi012.stdout:307 2023-07-08T12:40:56.583 INFO:teuthology.orchestra.run.smithi012.stdout:307.out 2023-07-08T12:40:56.583 INFO:teuthology.orchestra.run.smithi012.stdout:308 2023-07-08T12:40:56.583 INFO:teuthology.orchestra.run.smithi012.stdout:308.out 2023-07-08T12:40:56.583 INFO:teuthology.orchestra.run.smithi012.stdout:309 2023-07-08T12:40:56.583 INFO:teuthology.orchestra.run.smithi012.stdout:309.out 2023-07-08T12:40:56.583 INFO:teuthology.orchestra.run.smithi012.stdout:310 2023-07-08T12:40:56.584 INFO:teuthology.orchestra.run.smithi012.stdout:310.out 2023-07-08T12:40:56.584 INFO:teuthology.orchestra.run.smithi012.stdout:311 2023-07-08T12:40:56.584 INFO:teuthology.orchestra.run.smithi012.stdout:311.out 2023-07-08T12:40:56.584 INFO:teuthology.orchestra.run.smithi012.stdout:312 2023-07-08T12:40:56.584 INFO:teuthology.orchestra.run.smithi012.stdout:312.out 2023-07-08T12:40:56.584 INFO:teuthology.orchestra.run.smithi012.stdout:313 2023-07-08T12:40:56.584 INFO:teuthology.orchestra.run.smithi012.stdout:313.out 2023-07-08T12:40:56.584 INFO:teuthology.orchestra.run.smithi012.stdout:314 2023-07-08T12:40:56.585 INFO:teuthology.orchestra.run.smithi012.stdout:314.out 2023-07-08T12:40:56.585 INFO:teuthology.orchestra.run.smithi012.stdout:315 2023-07-08T12:40:56.585 INFO:teuthology.orchestra.run.smithi012.stdout:315.out 2023-07-08T12:40:56.585 INFO:teuthology.orchestra.run.smithi012.stdout:316 2023-07-08T12:40:56.585 INFO:teuthology.orchestra.run.smithi012.stdout:316.out 2023-07-08T12:40:56.585 INFO:teuthology.orchestra.run.smithi012.stdout:317 2023-07-08T12:40:56.585 INFO:teuthology.orchestra.run.smithi012.stdout:317.out 2023-07-08T12:40:56.585 INFO:teuthology.orchestra.run.smithi012.stdout:318 2023-07-08T12:40:56.586 INFO:teuthology.orchestra.run.smithi012.stdout:318.out 2023-07-08T12:40:56.586 INFO:teuthology.orchestra.run.smithi012.stdout:319 2023-07-08T12:40:56.586 INFO:teuthology.orchestra.run.smithi012.stdout:319.out 2023-07-08T12:40:56.586 INFO:teuthology.orchestra.run.smithi012.stdout:320 2023-07-08T12:40:56.586 INFO:teuthology.orchestra.run.smithi012.stdout:320.out 2023-07-08T12:40:56.586 INFO:teuthology.orchestra.run.smithi012.stdout:321 2023-07-08T12:40:56.586 INFO:teuthology.orchestra.run.smithi012.stdout:321.out 2023-07-08T12:40:56.586 INFO:teuthology.orchestra.run.smithi012.stdout:322 2023-07-08T12:40:56.587 INFO:teuthology.orchestra.run.smithi012.stdout:322.out 2023-07-08T12:40:56.587 INFO:teuthology.orchestra.run.smithi012.stdout:323 2023-07-08T12:40:56.587 INFO:teuthology.orchestra.run.smithi012.stdout:323.out 2023-07-08T12:40:56.587 INFO:teuthology.orchestra.run.smithi012.stdout:324 2023-07-08T12:40:56.587 INFO:teuthology.orchestra.run.smithi012.stdout:324.out 2023-07-08T12:40:56.587 INFO:teuthology.orchestra.run.smithi012.stdout:325 2023-07-08T12:40:56.587 INFO:teuthology.orchestra.run.smithi012.stdout:325.out 2023-07-08T12:40:56.587 INFO:teuthology.orchestra.run.smithi012.stdout:326 2023-07-08T12:40:56.588 INFO:teuthology.orchestra.run.smithi012.stdout:326.out 2023-07-08T12:40:56.588 INFO:teuthology.orchestra.run.smithi012.stdout:327 2023-07-08T12:40:56.588 INFO:teuthology.orchestra.run.smithi012.stdout:327.out 2023-07-08T12:40:56.588 INFO:teuthology.orchestra.run.smithi012.stdout:328 2023-07-08T12:40:56.588 INFO:teuthology.orchestra.run.smithi012.stdout:328.out 2023-07-08T12:40:56.588 INFO:teuthology.orchestra.run.smithi012.stdout:329 2023-07-08T12:40:56.588 INFO:teuthology.orchestra.run.smithi012.stdout:329.out 2023-07-08T12:40:56.588 INFO:teuthology.orchestra.run.smithi012.stdout:330 2023-07-08T12:40:56.589 INFO:teuthology.orchestra.run.smithi012.stdout:330.out 2023-07-08T12:40:56.589 INFO:teuthology.orchestra.run.smithi012.stdout:331 2023-07-08T12:40:56.589 INFO:teuthology.orchestra.run.smithi012.stdout:331.out 2023-07-08T12:40:56.589 INFO:teuthology.orchestra.run.smithi012.stdout:332 2023-07-08T12:40:56.589 INFO:teuthology.orchestra.run.smithi012.stdout:332.out 2023-07-08T12:40:56.589 INFO:teuthology.orchestra.run.smithi012.stdout:333 2023-07-08T12:40:56.589 INFO:teuthology.orchestra.run.smithi012.stdout:333.out 2023-07-08T12:40:56.589 INFO:teuthology.orchestra.run.smithi012.stdout:334 2023-07-08T12:40:56.590 INFO:teuthology.orchestra.run.smithi012.stdout:334.out 2023-07-08T12:40:56.590 INFO:teuthology.orchestra.run.smithi012.stdout:335 2023-07-08T12:40:56.590 INFO:teuthology.orchestra.run.smithi012.stdout:335.out 2023-07-08T12:40:56.590 INFO:teuthology.orchestra.run.smithi012.stdout:336 2023-07-08T12:40:56.590 INFO:teuthology.orchestra.run.smithi012.stdout:336.out 2023-07-08T12:40:56.590 INFO:teuthology.orchestra.run.smithi012.stdout:337 2023-07-08T12:40:56.590 INFO:teuthology.orchestra.run.smithi012.stdout:337.out 2023-07-08T12:40:56.590 INFO:teuthology.orchestra.run.smithi012.stdout:338 2023-07-08T12:40:56.591 INFO:teuthology.orchestra.run.smithi012.stdout:338.out 2023-07-08T12:40:56.591 INFO:teuthology.orchestra.run.smithi012.stdout:339 2023-07-08T12:40:56.591 INFO:teuthology.orchestra.run.smithi012.stdout:339.out 2023-07-08T12:40:56.591 INFO:teuthology.orchestra.run.smithi012.stdout:340 2023-07-08T12:40:56.591 INFO:teuthology.orchestra.run.smithi012.stdout:340.out 2023-07-08T12:40:56.591 INFO:teuthology.orchestra.run.smithi012.stdout:341 2023-07-08T12:40:56.591 INFO:teuthology.orchestra.run.smithi012.stdout:341.out 2023-07-08T12:40:56.591 INFO:teuthology.orchestra.run.smithi012.stdout:342 2023-07-08T12:40:56.592 INFO:teuthology.orchestra.run.smithi012.stdout:342.out 2023-07-08T12:40:56.592 INFO:teuthology.orchestra.run.smithi012.stdout:343 2023-07-08T12:40:56.592 INFO:teuthology.orchestra.run.smithi012.stdout:343.out 2023-07-08T12:40:56.592 INFO:teuthology.orchestra.run.smithi012.stdout:344 2023-07-08T12:40:56.592 INFO:teuthology.orchestra.run.smithi012.stdout:344.out 2023-07-08T12:40:56.592 INFO:teuthology.orchestra.run.smithi012.stdout:345 2023-07-08T12:40:56.592 INFO:teuthology.orchestra.run.smithi012.stdout:345.out 2023-07-08T12:40:56.592 INFO:teuthology.orchestra.run.smithi012.stdout:346 2023-07-08T12:40:56.593 INFO:teuthology.orchestra.run.smithi012.stdout:346.out 2023-07-08T12:40:56.593 INFO:teuthology.orchestra.run.smithi012.stdout:347 2023-07-08T12:40:56.593 INFO:teuthology.orchestra.run.smithi012.stdout:347.out 2023-07-08T12:40:56.593 INFO:teuthology.orchestra.run.smithi012.stdout:348 2023-07-08T12:40:56.593 INFO:teuthology.orchestra.run.smithi012.stdout:348.out 2023-07-08T12:40:56.593 INFO:teuthology.orchestra.run.smithi012.stdout:349 2023-07-08T12:40:56.593 INFO:teuthology.orchestra.run.smithi012.stdout:349.out 2023-07-08T12:40:56.593 INFO:teuthology.orchestra.run.smithi012.stdout:350 2023-07-08T12:40:56.594 INFO:teuthology.orchestra.run.smithi012.stdout:350.out 2023-07-08T12:40:56.594 INFO:teuthology.orchestra.run.smithi012.stdout:351 2023-07-08T12:40:56.594 INFO:teuthology.orchestra.run.smithi012.stdout:351.out 2023-07-08T12:40:56.594 INFO:teuthology.orchestra.run.smithi012.stdout:352 2023-07-08T12:40:56.594 INFO:teuthology.orchestra.run.smithi012.stdout:352.out 2023-07-08T12:40:56.594 INFO:teuthology.orchestra.run.smithi012.stdout:353 2023-07-08T12:40:56.594 INFO:teuthology.orchestra.run.smithi012.stdout:353.out 2023-07-08T12:40:56.594 INFO:teuthology.orchestra.run.smithi012.stdout:354 2023-07-08T12:40:56.595 INFO:teuthology.orchestra.run.smithi012.stdout:354.out 2023-07-08T12:40:56.595 INFO:teuthology.orchestra.run.smithi012.stdout:355 2023-07-08T12:40:56.595 INFO:teuthology.orchestra.run.smithi012.stdout:355.out 2023-07-08T12:40:56.595 INFO:teuthology.orchestra.run.smithi012.stdout:356 2023-07-08T12:40:56.595 INFO:teuthology.orchestra.run.smithi012.stdout:356.out 2023-07-08T12:40:56.595 INFO:teuthology.orchestra.run.smithi012.stdout:357 2023-07-08T12:40:56.595 INFO:teuthology.orchestra.run.smithi012.stdout:357.out 2023-07-08T12:40:56.595 INFO:teuthology.orchestra.run.smithi012.stdout:358 2023-07-08T12:40:56.596 INFO:teuthology.orchestra.run.smithi012.stdout:358.out 2023-07-08T12:40:56.596 INFO:teuthology.orchestra.run.smithi012.stdout:359 2023-07-08T12:40:56.596 INFO:teuthology.orchestra.run.smithi012.stdout:359.out 2023-07-08T12:40:56.596 INFO:teuthology.orchestra.run.smithi012.stdout:360 2023-07-08T12:40:56.596 INFO:teuthology.orchestra.run.smithi012.stdout:360.out 2023-07-08T12:40:56.596 INFO:teuthology.orchestra.run.smithi012.stdout:361 2023-07-08T12:40:56.596 INFO:teuthology.orchestra.run.smithi012.stdout:361.out 2023-07-08T12:40:56.596 INFO:teuthology.orchestra.run.smithi012.stdout:362 2023-07-08T12:40:56.597 INFO:teuthology.orchestra.run.smithi012.stdout:362.out 2023-07-08T12:40:56.597 INFO:teuthology.orchestra.run.smithi012.stdout:363 2023-07-08T12:40:56.597 INFO:teuthology.orchestra.run.smithi012.stdout:363.out 2023-07-08T12:40:56.597 INFO:teuthology.orchestra.run.smithi012.stdout:364 2023-07-08T12:40:56.597 INFO:teuthology.orchestra.run.smithi012.stdout:364.out 2023-07-08T12:40:56.597 INFO:teuthology.orchestra.run.smithi012.stdout:365 2023-07-08T12:40:56.597 INFO:teuthology.orchestra.run.smithi012.stdout:365.out 2023-07-08T12:40:56.597 INFO:teuthology.orchestra.run.smithi012.stdout:366 2023-07-08T12:40:56.598 INFO:teuthology.orchestra.run.smithi012.stdout:366.out 2023-07-08T12:40:56.598 INFO:teuthology.orchestra.run.smithi012.stdout:367 2023-07-08T12:40:56.598 INFO:teuthology.orchestra.run.smithi012.stdout:367.out 2023-07-08T12:40:56.598 INFO:teuthology.orchestra.run.smithi012.stdout:368 2023-07-08T12:40:56.598 INFO:teuthology.orchestra.run.smithi012.stdout:368.out 2023-07-08T12:40:56.598 INFO:teuthology.orchestra.run.smithi012.stdout:369 2023-07-08T12:40:56.598 INFO:teuthology.orchestra.run.smithi012.stdout:369.out 2023-07-08T12:40:56.598 INFO:teuthology.orchestra.run.smithi012.stdout:370 2023-07-08T12:40:56.599 INFO:teuthology.orchestra.run.smithi012.stdout:370.out 2023-07-08T12:40:56.599 INFO:teuthology.orchestra.run.smithi012.stdout:371 2023-07-08T12:40:56.599 INFO:teuthology.orchestra.run.smithi012.stdout:371.out 2023-07-08T12:40:56.599 INFO:teuthology.orchestra.run.smithi012.stdout:372 2023-07-08T12:40:56.599 INFO:teuthology.orchestra.run.smithi012.stdout:372.out 2023-07-08T12:40:56.599 INFO:teuthology.orchestra.run.smithi012.stdout:373 2023-07-08T12:40:56.599 INFO:teuthology.orchestra.run.smithi012.stdout:373.out 2023-07-08T12:40:56.599 INFO:teuthology.orchestra.run.smithi012.stdout:374 2023-07-08T12:40:56.600 INFO:teuthology.orchestra.run.smithi012.stdout:374.out 2023-07-08T12:40:56.600 INFO:teuthology.orchestra.run.smithi012.stdout:375 2023-07-08T12:40:56.600 INFO:teuthology.orchestra.run.smithi012.stdout:375.out 2023-07-08T12:40:56.600 INFO:teuthology.orchestra.run.smithi012.stdout:376 2023-07-08T12:40:56.600 INFO:teuthology.orchestra.run.smithi012.stdout:376.out 2023-07-08T12:40:56.600 INFO:teuthology.orchestra.run.smithi012.stdout:377 2023-07-08T12:40:56.600 INFO:teuthology.orchestra.run.smithi012.stdout:377.out 2023-07-08T12:40:56.601 INFO:teuthology.orchestra.run.smithi012.stdout:378 2023-07-08T12:40:56.601 INFO:teuthology.orchestra.run.smithi012.stdout:378.out 2023-07-08T12:40:56.601 INFO:teuthology.orchestra.run.smithi012.stdout:379 2023-07-08T12:40:56.601 INFO:teuthology.orchestra.run.smithi012.stdout:379.out 2023-07-08T12:40:56.601 INFO:teuthology.orchestra.run.smithi012.stdout:380 2023-07-08T12:40:56.601 INFO:teuthology.orchestra.run.smithi012.stdout:380.out 2023-07-08T12:40:56.601 INFO:teuthology.orchestra.run.smithi012.stdout:381 2023-07-08T12:40:56.601 INFO:teuthology.orchestra.run.smithi012.stdout:381.out 2023-07-08T12:40:56.602 INFO:teuthology.orchestra.run.smithi012.stdout:382 2023-07-08T12:40:56.602 INFO:teuthology.orchestra.run.smithi012.stdout:382.out 2023-07-08T12:40:56.602 INFO:teuthology.orchestra.run.smithi012.stdout:383 2023-07-08T12:40:56.602 INFO:teuthology.orchestra.run.smithi012.stdout:383.out 2023-07-08T12:40:56.602 INFO:teuthology.orchestra.run.smithi012.stdout:384 2023-07-08T12:40:56.602 INFO:teuthology.orchestra.run.smithi012.stdout:384.out 2023-07-08T12:40:56.602 INFO:teuthology.orchestra.run.smithi012.stdout:385 2023-07-08T12:40:56.602 INFO:teuthology.orchestra.run.smithi012.stdout:385.out 2023-07-08T12:40:56.603 INFO:teuthology.orchestra.run.smithi012.stdout:386 2023-07-08T12:40:56.603 INFO:teuthology.orchestra.run.smithi012.stdout:386.out 2023-07-08T12:40:56.603 INFO:teuthology.orchestra.run.smithi012.stdout:387 2023-07-08T12:40:56.603 INFO:teuthology.orchestra.run.smithi012.stdout:387.out 2023-07-08T12:40:56.603 INFO:teuthology.orchestra.run.smithi012.stdout:388 2023-07-08T12:40:56.603 INFO:teuthology.orchestra.run.smithi012.stdout:388.out 2023-07-08T12:40:56.603 INFO:teuthology.orchestra.run.smithi012.stdout:389 2023-07-08T12:40:56.603 INFO:teuthology.orchestra.run.smithi012.stdout:389.out 2023-07-08T12:40:56.604 INFO:teuthology.orchestra.run.smithi012.stdout:390 2023-07-08T12:40:56.604 INFO:teuthology.orchestra.run.smithi012.stdout:390.out 2023-07-08T12:40:56.604 INFO:teuthology.orchestra.run.smithi012.stdout:391 2023-07-08T12:40:56.604 INFO:teuthology.orchestra.run.smithi012.stdout:391.out 2023-07-08T12:40:56.604 INFO:teuthology.orchestra.run.smithi012.stdout:392 2023-07-08T12:40:56.604 INFO:teuthology.orchestra.run.smithi012.stdout:392.out 2023-07-08T12:40:56.604 INFO:teuthology.orchestra.run.smithi012.stdout:393 2023-07-08T12:40:56.604 INFO:teuthology.orchestra.run.smithi012.stdout:393.out 2023-07-08T12:40:56.605 INFO:teuthology.orchestra.run.smithi012.stdout:394 2023-07-08T12:40:56.605 INFO:teuthology.orchestra.run.smithi012.stdout:394.out 2023-07-08T12:40:56.605 INFO:teuthology.orchestra.run.smithi012.stdout:395 2023-07-08T12:40:56.605 INFO:teuthology.orchestra.run.smithi012.stdout:395.out 2023-07-08T12:40:56.605 INFO:teuthology.orchestra.run.smithi012.stdout:396 2023-07-08T12:40:56.605 INFO:teuthology.orchestra.run.smithi012.stdout:396.out 2023-07-08T12:40:56.605 INFO:teuthology.orchestra.run.smithi012.stdout:397 2023-07-08T12:40:56.605 INFO:teuthology.orchestra.run.smithi012.stdout:397.out 2023-07-08T12:40:56.606 INFO:teuthology.orchestra.run.smithi012.stdout:398 2023-07-08T12:40:56.606 INFO:teuthology.orchestra.run.smithi012.stdout:398.out 2023-07-08T12:40:56.606 INFO:teuthology.orchestra.run.smithi012.stdout:399 2023-07-08T12:40:56.606 INFO:teuthology.orchestra.run.smithi012.stdout:399.out 2023-07-08T12:40:56.606 INFO:teuthology.orchestra.run.smithi012.stdout:400 2023-07-08T12:40:56.606 INFO:teuthology.orchestra.run.smithi012.stdout:400.out 2023-07-08T12:40:56.606 INFO:teuthology.orchestra.run.smithi012.stdout:401 2023-07-08T12:40:56.606 INFO:teuthology.orchestra.run.smithi012.stdout:401.out 2023-07-08T12:40:56.607 INFO:teuthology.orchestra.run.smithi012.stdout:402 2023-07-08T12:40:56.607 INFO:teuthology.orchestra.run.smithi012.stdout:402.out 2023-07-08T12:40:56.607 INFO:teuthology.orchestra.run.smithi012.stdout:403 2023-07-08T12:40:56.607 INFO:teuthology.orchestra.run.smithi012.stdout:403.out 2023-07-08T12:40:56.607 INFO:teuthology.orchestra.run.smithi012.stdout:404 2023-07-08T12:40:56.607 INFO:teuthology.orchestra.run.smithi012.stdout:404.out 2023-07-08T12:40:56.607 INFO:teuthology.orchestra.run.smithi012.stdout:405 2023-07-08T12:40:56.607 INFO:teuthology.orchestra.run.smithi012.stdout:405.out 2023-07-08T12:40:56.608 INFO:teuthology.orchestra.run.smithi012.stdout:406 2023-07-08T12:40:56.608 INFO:teuthology.orchestra.run.smithi012.stdout:406.out 2023-07-08T12:40:56.608 INFO:teuthology.orchestra.run.smithi012.stdout:407 2023-07-08T12:40:56.608 INFO:teuthology.orchestra.run.smithi012.stdout:407.out 2023-07-08T12:40:56.608 INFO:teuthology.orchestra.run.smithi012.stdout:408 2023-07-08T12:40:56.608 INFO:teuthology.orchestra.run.smithi012.stdout:408.out 2023-07-08T12:40:56.608 INFO:teuthology.orchestra.run.smithi012.stdout:409 2023-07-08T12:40:56.608 INFO:teuthology.orchestra.run.smithi012.stdout:409.out 2023-07-08T12:40:56.609 INFO:teuthology.orchestra.run.smithi012.stdout:410 2023-07-08T12:40:56.609 INFO:teuthology.orchestra.run.smithi012.stdout:410.out 2023-07-08T12:40:56.609 INFO:teuthology.orchestra.run.smithi012.stdout:411 2023-07-08T12:40:56.609 INFO:teuthology.orchestra.run.smithi012.stdout:411.out 2023-07-08T12:40:56.609 INFO:teuthology.orchestra.run.smithi012.stdout:412 2023-07-08T12:40:56.609 INFO:teuthology.orchestra.run.smithi012.stdout:412.out 2023-07-08T12:40:56.609 INFO:teuthology.orchestra.run.smithi012.stdout:413 2023-07-08T12:40:56.609 INFO:teuthology.orchestra.run.smithi012.stdout:413.out 2023-07-08T12:40:56.610 INFO:teuthology.orchestra.run.smithi012.stdout:414 2023-07-08T12:40:56.610 INFO:teuthology.orchestra.run.smithi012.stdout:414.out 2023-07-08T12:40:56.610 INFO:teuthology.orchestra.run.smithi012.stdout:415 2023-07-08T12:40:56.610 INFO:teuthology.orchestra.run.smithi012.stdout:415.out 2023-07-08T12:40:56.610 INFO:teuthology.orchestra.run.smithi012.stdout:416 2023-07-08T12:40:56.610 INFO:teuthology.orchestra.run.smithi012.stdout:416.out 2023-07-08T12:40:56.610 INFO:teuthology.orchestra.run.smithi012.stdout:417 2023-07-08T12:40:56.610 INFO:teuthology.orchestra.run.smithi012.stdout:417.out 2023-07-08T12:40:56.611 INFO:teuthology.orchestra.run.smithi012.stdout:418 2023-07-08T12:40:56.611 INFO:teuthology.orchestra.run.smithi012.stdout:418.out 2023-07-08T12:40:56.611 INFO:teuthology.orchestra.run.smithi012.stdout:419 2023-07-08T12:40:56.611 INFO:teuthology.orchestra.run.smithi012.stdout:419.out 2023-07-08T12:40:56.611 INFO:teuthology.orchestra.run.smithi012.stdout:420 2023-07-08T12:40:56.611 INFO:teuthology.orchestra.run.smithi012.stdout:420.out 2023-07-08T12:40:56.611 INFO:teuthology.orchestra.run.smithi012.stdout:421 2023-07-08T12:40:56.612 INFO:teuthology.orchestra.run.smithi012.stdout:421.out 2023-07-08T12:40:56.612 INFO:teuthology.orchestra.run.smithi012.stdout:422 2023-07-08T12:40:56.612 INFO:teuthology.orchestra.run.smithi012.stdout:422.out 2023-07-08T12:40:56.612 INFO:teuthology.orchestra.run.smithi012.stdout:423 2023-07-08T12:40:56.612 INFO:teuthology.orchestra.run.smithi012.stdout:423.out 2023-07-08T12:40:56.612 INFO:teuthology.orchestra.run.smithi012.stdout:424 2023-07-08T12:40:56.612 INFO:teuthology.orchestra.run.smithi012.stdout:424.out 2023-07-08T12:40:56.612 INFO:teuthology.orchestra.run.smithi012.stdout:425 2023-07-08T12:40:56.613 INFO:teuthology.orchestra.run.smithi012.stdout:425.out 2023-07-08T12:40:56.613 INFO:teuthology.orchestra.run.smithi012.stdout:426 2023-07-08T12:40:56.613 INFO:teuthology.orchestra.run.smithi012.stdout:426.out 2023-07-08T12:40:56.613 INFO:teuthology.orchestra.run.smithi012.stdout:427 2023-07-08T12:40:56.613 INFO:teuthology.orchestra.run.smithi012.stdout:427.out 2023-07-08T12:40:56.613 INFO:teuthology.orchestra.run.smithi012.stdout:428 2023-07-08T12:40:56.613 INFO:teuthology.orchestra.run.smithi012.stdout:428.out 2023-07-08T12:40:56.613 INFO:teuthology.orchestra.run.smithi012.stdout:429 2023-07-08T12:40:56.614 INFO:teuthology.orchestra.run.smithi012.stdout:429.out 2023-07-08T12:40:56.614 INFO:teuthology.orchestra.run.smithi012.stdout:430 2023-07-08T12:40:56.614 INFO:teuthology.orchestra.run.smithi012.stdout:430.out 2023-07-08T12:40:56.614 INFO:teuthology.orchestra.run.smithi012.stdout:431 2023-07-08T12:40:56.614 INFO:teuthology.orchestra.run.smithi012.stdout:431.out 2023-07-08T12:40:56.614 INFO:teuthology.orchestra.run.smithi012.stdout:432 2023-07-08T12:40:56.614 INFO:teuthology.orchestra.run.smithi012.stdout:432.out 2023-07-08T12:40:56.614 INFO:teuthology.orchestra.run.smithi012.stdout:433 2023-07-08T12:40:56.615 INFO:teuthology.orchestra.run.smithi012.stdout:433.out 2023-07-08T12:40:56.615 INFO:teuthology.orchestra.run.smithi012.stdout:434 2023-07-08T12:40:56.615 INFO:teuthology.orchestra.run.smithi012.stdout:434.out 2023-07-08T12:40:56.615 INFO:teuthology.orchestra.run.smithi012.stdout:435 2023-07-08T12:40:56.615 INFO:teuthology.orchestra.run.smithi012.stdout:435.out 2023-07-08T12:40:56.615 INFO:teuthology.orchestra.run.smithi012.stdout:436 2023-07-08T12:40:56.615 INFO:teuthology.orchestra.run.smithi012.stdout:436.out 2023-07-08T12:40:56.615 INFO:teuthology.orchestra.run.smithi012.stdout:437 2023-07-08T12:40:56.616 INFO:teuthology.orchestra.run.smithi012.stdout:437.out 2023-07-08T12:40:56.616 INFO:teuthology.orchestra.run.smithi012.stdout:438 2023-07-08T12:40:56.616 INFO:teuthology.orchestra.run.smithi012.stdout:438.out 2023-07-08T12:40:56.616 INFO:teuthology.orchestra.run.smithi012.stdout:439 2023-07-08T12:40:56.616 INFO:teuthology.orchestra.run.smithi012.stdout:439.out 2023-07-08T12:40:56.616 INFO:teuthology.orchestra.run.smithi012.stdout:440 2023-07-08T12:40:56.616 INFO:teuthology.orchestra.run.smithi012.stdout:440.out 2023-07-08T12:40:56.616 INFO:teuthology.orchestra.run.smithi012.stdout:441 2023-07-08T12:40:56.617 INFO:teuthology.orchestra.run.smithi012.stdout:441.out 2023-07-08T12:40:56.617 INFO:teuthology.orchestra.run.smithi012.stdout:442 2023-07-08T12:40:56.617 INFO:teuthology.orchestra.run.smithi012.stdout:442.out 2023-07-08T12:40:56.617 INFO:teuthology.orchestra.run.smithi012.stdout:443 2023-07-08T12:40:56.617 INFO:teuthology.orchestra.run.smithi012.stdout:443.out 2023-07-08T12:40:56.617 INFO:teuthology.orchestra.run.smithi012.stdout:444 2023-07-08T12:40:56.617 INFO:teuthology.orchestra.run.smithi012.stdout:444.out 2023-07-08T12:40:56.617 INFO:teuthology.orchestra.run.smithi012.stdout:445 2023-07-08T12:40:56.618 INFO:teuthology.orchestra.run.smithi012.stdout:445.out 2023-07-08T12:40:56.618 INFO:teuthology.orchestra.run.smithi012.stdout:446 2023-07-08T12:40:56.618 INFO:teuthology.orchestra.run.smithi012.stdout:446.out 2023-07-08T12:40:56.618 INFO:teuthology.orchestra.run.smithi012.stdout:447 2023-07-08T12:40:56.618 INFO:teuthology.orchestra.run.smithi012.stdout:447.out 2023-07-08T12:40:56.618 INFO:teuthology.orchestra.run.smithi012.stdout:448 2023-07-08T12:40:56.618 INFO:teuthology.orchestra.run.smithi012.stdout:448.out 2023-07-08T12:40:56.618 INFO:teuthology.orchestra.run.smithi012.stdout:449 2023-07-08T12:40:56.619 INFO:teuthology.orchestra.run.smithi012.stdout:449.out 2023-07-08T12:40:56.619 INFO:teuthology.orchestra.run.smithi012.stdout:450 2023-07-08T12:40:56.619 INFO:teuthology.orchestra.run.smithi012.stdout:450.out 2023-07-08T12:40:56.619 INFO:teuthology.orchestra.run.smithi012.stdout:451 2023-07-08T12:40:56.619 INFO:teuthology.orchestra.run.smithi012.stdout:451.out 2023-07-08T12:40:56.619 INFO:teuthology.orchestra.run.smithi012.stdout:452 2023-07-08T12:40:56.619 INFO:teuthology.orchestra.run.smithi012.stdout:452.out 2023-07-08T12:40:56.619 INFO:teuthology.orchestra.run.smithi012.stdout:453 2023-07-08T12:40:56.620 INFO:teuthology.orchestra.run.smithi012.stdout:453.out 2023-07-08T12:40:56.620 INFO:teuthology.orchestra.run.smithi012.stdout:454 2023-07-08T12:40:56.620 INFO:teuthology.orchestra.run.smithi012.stdout:454.out 2023-07-08T12:40:56.620 INFO:teuthology.orchestra.run.smithi012.stdout:455 2023-07-08T12:40:56.620 INFO:teuthology.orchestra.run.smithi012.stdout:455.out 2023-07-08T12:40:56.620 INFO:teuthology.orchestra.run.smithi012.stdout:456 2023-07-08T12:40:56.620 INFO:teuthology.orchestra.run.smithi012.stdout:456.out 2023-07-08T12:40:56.620 INFO:teuthology.orchestra.run.smithi012.stdout:457 2023-07-08T12:40:56.621 INFO:teuthology.orchestra.run.smithi012.stdout:457.out 2023-07-08T12:40:56.621 INFO:teuthology.orchestra.run.smithi012.stdout:458 2023-07-08T12:40:56.621 INFO:teuthology.orchestra.run.smithi012.stdout:458.out 2023-07-08T12:40:56.621 INFO:teuthology.orchestra.run.smithi012.stdout:459 2023-07-08T12:40:56.621 INFO:teuthology.orchestra.run.smithi012.stdout:459.out 2023-07-08T12:40:56.621 INFO:teuthology.orchestra.run.smithi012.stdout:460 2023-07-08T12:40:56.621 INFO:teuthology.orchestra.run.smithi012.stdout:460.out 2023-07-08T12:40:56.622 INFO:teuthology.orchestra.run.smithi012.stdout:461 2023-07-08T12:40:56.622 INFO:teuthology.orchestra.run.smithi012.stdout:461.out 2023-07-08T12:40:56.622 INFO:teuthology.orchestra.run.smithi012.stdout:462 2023-07-08T12:40:56.622 INFO:teuthology.orchestra.run.smithi012.stdout:462.out 2023-07-08T12:40:56.622 INFO:teuthology.orchestra.run.smithi012.stdout:463 2023-07-08T12:40:56.622 INFO:teuthology.orchestra.run.smithi012.stdout:463.out 2023-07-08T12:40:56.622 INFO:teuthology.orchestra.run.smithi012.stdout:464 2023-07-08T12:40:56.622 INFO:teuthology.orchestra.run.smithi012.stdout:464.out 2023-07-08T12:40:56.623 INFO:teuthology.orchestra.run.smithi012.stdout:465 2023-07-08T12:40:56.623 INFO:teuthology.orchestra.run.smithi012.stdout:465.out 2023-07-08T12:40:56.623 INFO:teuthology.orchestra.run.smithi012.stdout:466 2023-07-08T12:40:56.623 INFO:teuthology.orchestra.run.smithi012.stdout:466.out 2023-07-08T12:40:56.623 INFO:teuthology.orchestra.run.smithi012.stdout:467 2023-07-08T12:40:56.623 INFO:teuthology.orchestra.run.smithi012.stdout:467.out 2023-07-08T12:40:56.623 INFO:teuthology.orchestra.run.smithi012.stdout:468 2023-07-08T12:40:56.623 INFO:teuthology.orchestra.run.smithi012.stdout:468.out 2023-07-08T12:40:56.624 INFO:teuthology.orchestra.run.smithi012.stdout:469 2023-07-08T12:40:56.624 INFO:teuthology.orchestra.run.smithi012.stdout:469.out 2023-07-08T12:40:56.624 INFO:teuthology.orchestra.run.smithi012.stdout:470 2023-07-08T12:40:56.624 INFO:teuthology.orchestra.run.smithi012.stdout:470.out 2023-07-08T12:40:56.624 INFO:teuthology.orchestra.run.smithi012.stdout:471 2023-07-08T12:40:56.624 INFO:teuthology.orchestra.run.smithi012.stdout:471.out 2023-07-08T12:40:56.624 INFO:teuthology.orchestra.run.smithi012.stdout:472 2023-07-08T12:40:56.624 INFO:teuthology.orchestra.run.smithi012.stdout:472.out 2023-07-08T12:40:56.625 INFO:teuthology.orchestra.run.smithi012.stdout:473 2023-07-08T12:40:56.625 INFO:teuthology.orchestra.run.smithi012.stdout:473.out 2023-07-08T12:40:56.625 INFO:teuthology.orchestra.run.smithi012.stdout:474 2023-07-08T12:40:56.625 INFO:teuthology.orchestra.run.smithi012.stdout:474.out 2023-07-08T12:40:56.625 INFO:teuthology.orchestra.run.smithi012.stdout:475 2023-07-08T12:40:56.625 INFO:teuthology.orchestra.run.smithi012.stdout:475.out 2023-07-08T12:40:56.625 INFO:teuthology.orchestra.run.smithi012.stdout:476 2023-07-08T12:40:56.625 INFO:teuthology.orchestra.run.smithi012.stdout:476.out 2023-07-08T12:40:56.626 INFO:teuthology.orchestra.run.smithi012.stdout:477 2023-07-08T12:40:56.626 INFO:teuthology.orchestra.run.smithi012.stdout:477.out 2023-07-08T12:40:56.626 INFO:teuthology.orchestra.run.smithi012.stdout:478 2023-07-08T12:40:56.626 INFO:teuthology.orchestra.run.smithi012.stdout:478.out 2023-07-08T12:40:56.626 INFO:teuthology.orchestra.run.smithi012.stdout:479 2023-07-08T12:40:56.626 INFO:teuthology.orchestra.run.smithi012.stdout:479.out 2023-07-08T12:40:56.626 INFO:teuthology.orchestra.run.smithi012.stdout:480 2023-07-08T12:40:56.626 INFO:teuthology.orchestra.run.smithi012.stdout:480.out 2023-07-08T12:40:56.627 INFO:teuthology.orchestra.run.smithi012.stdout:481 2023-07-08T12:40:56.627 INFO:teuthology.orchestra.run.smithi012.stdout:481.out 2023-07-08T12:40:56.627 INFO:teuthology.orchestra.run.smithi012.stdout:482 2023-07-08T12:40:56.627 INFO:teuthology.orchestra.run.smithi012.stdout:482.out 2023-07-08T12:40:56.627 INFO:teuthology.orchestra.run.smithi012.stdout:483 2023-07-08T12:40:56.627 INFO:teuthology.orchestra.run.smithi012.stdout:483.out 2023-07-08T12:40:56.627 INFO:teuthology.orchestra.run.smithi012.stdout:484 2023-07-08T12:40:56.627 INFO:teuthology.orchestra.run.smithi012.stdout:484.out 2023-07-08T12:40:56.628 INFO:teuthology.orchestra.run.smithi012.stdout:485 2023-07-08T12:40:56.628 INFO:teuthology.orchestra.run.smithi012.stdout:485.out 2023-07-08T12:40:56.628 INFO:teuthology.orchestra.run.smithi012.stdout:486 2023-07-08T12:40:56.628 INFO:teuthology.orchestra.run.smithi012.stdout:486.out 2023-07-08T12:40:56.628 INFO:teuthology.orchestra.run.smithi012.stdout:487 2023-07-08T12:40:56.628 INFO:teuthology.orchestra.run.smithi012.stdout:487.out 2023-07-08T12:40:56.628 INFO:teuthology.orchestra.run.smithi012.stdout:488 2023-07-08T12:40:56.628 INFO:teuthology.orchestra.run.smithi012.stdout:488.out 2023-07-08T12:40:56.629 INFO:teuthology.orchestra.run.smithi012.stdout:489 2023-07-08T12:40:56.629 INFO:teuthology.orchestra.run.smithi012.stdout:489.out 2023-07-08T12:40:56.629 INFO:teuthology.orchestra.run.smithi012.stdout:490 2023-07-08T12:40:56.629 INFO:teuthology.orchestra.run.smithi012.stdout:490.out 2023-07-08T12:40:56.629 INFO:teuthology.orchestra.run.smithi012.stdout:491 2023-07-08T12:40:56.629 INFO:teuthology.orchestra.run.smithi012.stdout:491.out 2023-07-08T12:40:56.629 INFO:teuthology.orchestra.run.smithi012.stdout:492 2023-07-08T12:40:56.629 INFO:teuthology.orchestra.run.smithi012.stdout:492.out 2023-07-08T12:40:56.630 INFO:teuthology.orchestra.run.smithi012.stdout:493 2023-07-08T12:40:56.630 INFO:teuthology.orchestra.run.smithi012.stdout:493.out 2023-07-08T12:40:56.630 INFO:teuthology.orchestra.run.smithi012.stdout:494 2023-07-08T12:40:56.630 INFO:teuthology.orchestra.run.smithi012.stdout:494.out 2023-07-08T12:40:56.630 INFO:teuthology.orchestra.run.smithi012.stdout:495 2023-07-08T12:40:56.630 INFO:teuthology.orchestra.run.smithi012.stdout:495.out 2023-07-08T12:40:56.630 INFO:teuthology.orchestra.run.smithi012.stdout:496 2023-07-08T12:40:56.630 INFO:teuthology.orchestra.run.smithi012.stdout:496.out 2023-07-08T12:40:56.631 INFO:teuthology.orchestra.run.smithi012.stdout:497 2023-07-08T12:40:56.631 INFO:teuthology.orchestra.run.smithi012.stdout:497.out 2023-07-08T12:40:56.631 INFO:teuthology.orchestra.run.smithi012.stdout:498 2023-07-08T12:40:56.631 INFO:teuthology.orchestra.run.smithi012.stdout:498.out 2023-07-08T12:40:56.631 INFO:teuthology.orchestra.run.smithi012.stdout:499 2023-07-08T12:40:56.631 INFO:teuthology.orchestra.run.smithi012.stdout:499.out 2023-07-08T12:40:56.631 INFO:teuthology.orchestra.run.smithi012.stdout:500 2023-07-08T12:40:56.631 INFO:teuthology.orchestra.run.smithi012.stdout:500.out 2023-07-08T12:40:56.632 INFO:teuthology.orchestra.run.smithi012.stdout:501 2023-07-08T12:40:56.632 INFO:teuthology.orchestra.run.smithi012.stdout:501.out 2023-07-08T12:40:56.632 INFO:teuthology.orchestra.run.smithi012.stdout:502 2023-07-08T12:40:56.632 INFO:teuthology.orchestra.run.smithi012.stdout:502.out 2023-07-08T12:40:56.632 INFO:teuthology.orchestra.run.smithi012.stdout:503 2023-07-08T12:40:56.632 INFO:teuthology.orchestra.run.smithi012.stdout:503.out 2023-07-08T12:40:56.632 INFO:teuthology.orchestra.run.smithi012.stdout:504 2023-07-08T12:40:56.633 INFO:teuthology.orchestra.run.smithi012.stdout:504.out 2023-07-08T12:40:56.633 INFO:teuthology.orchestra.run.smithi012.stdout:505 2023-07-08T12:40:56.633 INFO:teuthology.orchestra.run.smithi012.stdout:505.out 2023-07-08T12:40:56.633 INFO:teuthology.orchestra.run.smithi012.stdout:506 2023-07-08T12:40:56.633 INFO:teuthology.orchestra.run.smithi012.stdout:506.out 2023-07-08T12:40:56.633 INFO:teuthology.orchestra.run.smithi012.stdout:507 2023-07-08T12:40:56.633 INFO:teuthology.orchestra.run.smithi012.stdout:507.out 2023-07-08T12:40:56.633 INFO:teuthology.orchestra.run.smithi012.stdout:508 2023-07-08T12:40:56.634 INFO:teuthology.orchestra.run.smithi012.stdout:508.out 2023-07-08T12:40:56.634 INFO:teuthology.orchestra.run.smithi012.stdout:509 2023-07-08T12:40:56.634 INFO:teuthology.orchestra.run.smithi012.stdout:509.out 2023-07-08T12:40:56.634 INFO:teuthology.orchestra.run.smithi012.stdout:510 2023-07-08T12:40:56.634 INFO:teuthology.orchestra.run.smithi012.stdout:510.out 2023-07-08T12:40:56.634 INFO:teuthology.orchestra.run.smithi012.stdout:511 2023-07-08T12:40:56.634 INFO:teuthology.orchestra.run.smithi012.stdout:511.out 2023-07-08T12:40:56.634 INFO:teuthology.orchestra.run.smithi012.stdout:512 2023-07-08T12:40:56.634 INFO:teuthology.orchestra.run.smithi012.stdout:512.out 2023-07-08T12:40:56.635 INFO:teuthology.orchestra.run.smithi012.stdout:513 2023-07-08T12:40:56.635 INFO:teuthology.orchestra.run.smithi012.stdout:513.out 2023-07-08T12:40:56.635 INFO:teuthology.orchestra.run.smithi012.stdout:514 2023-07-08T12:40:56.635 INFO:teuthology.orchestra.run.smithi012.stdout:514.out 2023-07-08T12:40:56.635 INFO:teuthology.orchestra.run.smithi012.stdout:515 2023-07-08T12:40:56.635 INFO:teuthology.orchestra.run.smithi012.stdout:515.out 2023-07-08T12:40:56.635 INFO:teuthology.orchestra.run.smithi012.stdout:516 2023-07-08T12:40:56.636 INFO:teuthology.orchestra.run.smithi012.stdout:516.out 2023-07-08T12:40:56.636 INFO:teuthology.orchestra.run.smithi012.stdout:517 2023-07-08T12:40:56.636 INFO:teuthology.orchestra.run.smithi012.stdout:517.out 2023-07-08T12:40:56.636 INFO:teuthology.orchestra.run.smithi012.stdout:518 2023-07-08T12:40:56.636 INFO:teuthology.orchestra.run.smithi012.stdout:518.out 2023-07-08T12:40:56.636 INFO:teuthology.orchestra.run.smithi012.stdout:519 2023-07-08T12:40:56.636 INFO:teuthology.orchestra.run.smithi012.stdout:519.out 2023-07-08T12:40:56.636 INFO:teuthology.orchestra.run.smithi012.stdout:520 2023-07-08T12:40:56.637 INFO:teuthology.orchestra.run.smithi012.stdout:520.out 2023-07-08T12:40:56.637 INFO:teuthology.orchestra.run.smithi012.stdout:521 2023-07-08T12:40:56.637 INFO:teuthology.orchestra.run.smithi012.stdout:521.out 2023-07-08T12:40:56.637 INFO:teuthology.orchestra.run.smithi012.stdout:522 2023-07-08T12:40:56.637 INFO:teuthology.orchestra.run.smithi012.stdout:522.out 2023-07-08T12:40:56.637 INFO:teuthology.orchestra.run.smithi012.stdout:523 2023-07-08T12:40:56.637 INFO:teuthology.orchestra.run.smithi012.stdout:523.out 2023-07-08T12:40:56.637 INFO:teuthology.orchestra.run.smithi012.stdout:524 2023-07-08T12:40:56.638 INFO:teuthology.orchestra.run.smithi012.stdout:524.out 2023-07-08T12:40:56.638 INFO:teuthology.orchestra.run.smithi012.stdout:525 2023-07-08T12:40:56.638 INFO:teuthology.orchestra.run.smithi012.stdout:525.out 2023-07-08T12:40:56.638 INFO:teuthology.orchestra.run.smithi012.stdout:526 2023-07-08T12:40:56.638 INFO:teuthology.orchestra.run.smithi012.stdout:526.out 2023-07-08T12:40:56.638 INFO:teuthology.orchestra.run.smithi012.stdout:527 2023-07-08T12:40:56.638 INFO:teuthology.orchestra.run.smithi012.stdout:527.out 2023-07-08T12:40:56.638 INFO:teuthology.orchestra.run.smithi012.stdout:528 2023-07-08T12:40:56.639 INFO:teuthology.orchestra.run.smithi012.stdout:528.out 2023-07-08T12:40:56.639 INFO:teuthology.orchestra.run.smithi012.stdout:529 2023-07-08T12:40:56.639 INFO:teuthology.orchestra.run.smithi012.stdout:529.out 2023-07-08T12:40:56.639 INFO:teuthology.orchestra.run.smithi012.stdout:530 2023-07-08T12:40:56.639 INFO:teuthology.orchestra.run.smithi012.stdout:530.out 2023-07-08T12:40:56.639 INFO:teuthology.orchestra.run.smithi012.stdout:531 2023-07-08T12:40:56.639 INFO:teuthology.orchestra.run.smithi012.stdout:531.out 2023-07-08T12:40:56.639 INFO:teuthology.orchestra.run.smithi012.stdout:532 2023-07-08T12:40:56.639 INFO:teuthology.orchestra.run.smithi012.stdout:532.out 2023-07-08T12:40:56.640 INFO:teuthology.orchestra.run.smithi012.stdout:533 2023-07-08T12:40:56.640 INFO:teuthology.orchestra.run.smithi012.stdout:533.out 2023-07-08T12:40:56.640 INFO:teuthology.orchestra.run.smithi012.stdout:534 2023-07-08T12:40:56.640 INFO:teuthology.orchestra.run.smithi012.stdout:534.out 2023-07-08T12:40:56.640 INFO:teuthology.orchestra.run.smithi012.stdout:535 2023-07-08T12:40:56.640 INFO:teuthology.orchestra.run.smithi012.stdout:535.out 2023-07-08T12:40:56.640 INFO:teuthology.orchestra.run.smithi012.stdout:536 2023-07-08T12:40:56.641 INFO:teuthology.orchestra.run.smithi012.stdout:536.out 2023-07-08T12:40:56.641 INFO:teuthology.orchestra.run.smithi012.stdout:537 2023-07-08T12:40:56.641 INFO:teuthology.orchestra.run.smithi012.stdout:537.out 2023-07-08T12:40:56.641 INFO:teuthology.orchestra.run.smithi012.stdout:538 2023-07-08T12:40:56.641 INFO:teuthology.orchestra.run.smithi012.stdout:538.out 2023-07-08T12:40:56.641 INFO:teuthology.orchestra.run.smithi012.stdout:539 2023-07-08T12:40:56.641 INFO:teuthology.orchestra.run.smithi012.stdout:539.out 2023-07-08T12:40:56.641 INFO:teuthology.orchestra.run.smithi012.stdout:540 2023-07-08T12:40:56.642 INFO:teuthology.orchestra.run.smithi012.stdout:540.out 2023-07-08T12:40:56.642 INFO:teuthology.orchestra.run.smithi012.stdout:541 2023-07-08T12:40:56.642 INFO:teuthology.orchestra.run.smithi012.stdout:541.out 2023-07-08T12:40:56.642 INFO:teuthology.orchestra.run.smithi012.stdout:542 2023-07-08T12:40:56.642 INFO:teuthology.orchestra.run.smithi012.stdout:542.out 2023-07-08T12:40:56.642 INFO:teuthology.orchestra.run.smithi012.stdout:543 2023-07-08T12:40:56.642 INFO:teuthology.orchestra.run.smithi012.stdout:543.out 2023-07-08T12:40:56.642 INFO:teuthology.orchestra.run.smithi012.stdout:544 2023-07-08T12:40:56.643 INFO:teuthology.orchestra.run.smithi012.stdout:544.out 2023-07-08T12:40:56.643 INFO:teuthology.orchestra.run.smithi012.stdout:545 2023-07-08T12:40:56.643 INFO:teuthology.orchestra.run.smithi012.stdout:545.out 2023-07-08T12:40:56.643 INFO:teuthology.orchestra.run.smithi012.stdout:546 2023-07-08T12:40:56.643 INFO:teuthology.orchestra.run.smithi012.stdout:546.out 2023-07-08T12:40:56.643 INFO:teuthology.orchestra.run.smithi012.stdout:547 2023-07-08T12:40:56.643 INFO:teuthology.orchestra.run.smithi012.stdout:547.out 2023-07-08T12:40:56.643 INFO:teuthology.orchestra.run.smithi012.stdout:548 2023-07-08T12:40:56.644 INFO:teuthology.orchestra.run.smithi012.stdout:548.out 2023-07-08T12:40:56.644 INFO:teuthology.orchestra.run.smithi012.stdout:549 2023-07-08T12:40:56.644 INFO:teuthology.orchestra.run.smithi012.stdout:549.out 2023-07-08T12:40:56.644 INFO:teuthology.orchestra.run.smithi012.stdout:550 2023-07-08T12:40:56.644 INFO:teuthology.orchestra.run.smithi012.stdout:550.out 2023-07-08T12:40:56.644 INFO:teuthology.orchestra.run.smithi012.stdout:551 2023-07-08T12:40:56.644 INFO:teuthology.orchestra.run.smithi012.stdout:551.out 2023-07-08T12:40:56.644 INFO:teuthology.orchestra.run.smithi012.stdout:552 2023-07-08T12:40:56.645 INFO:teuthology.orchestra.run.smithi012.stdout:552.out 2023-07-08T12:40:56.645 INFO:teuthology.orchestra.run.smithi012.stdout:553 2023-07-08T12:40:56.645 INFO:teuthology.orchestra.run.smithi012.stdout:553.out 2023-07-08T12:40:56.645 INFO:teuthology.orchestra.run.smithi012.stdout:554 2023-07-08T12:40:56.645 INFO:teuthology.orchestra.run.smithi012.stdout:554.out 2023-07-08T12:40:56.645 INFO:teuthology.orchestra.run.smithi012.stdout:555 2023-07-08T12:40:56.645 INFO:teuthology.orchestra.run.smithi012.stdout:555.out 2023-07-08T12:40:56.645 INFO:teuthology.orchestra.run.smithi012.stdout:556 2023-07-08T12:40:56.646 INFO:teuthology.orchestra.run.smithi012.stdout:556.out 2023-07-08T12:40:56.646 INFO:teuthology.orchestra.run.smithi012.stdout:557 2023-07-08T12:40:56.646 INFO:teuthology.orchestra.run.smithi012.stdout:557.out 2023-07-08T12:40:56.646 INFO:teuthology.orchestra.run.smithi012.stdout:558 2023-07-08T12:40:56.646 INFO:teuthology.orchestra.run.smithi012.stdout:558.out 2023-07-08T12:40:56.646 INFO:teuthology.orchestra.run.smithi012.stdout:559 2023-07-08T12:40:56.646 INFO:teuthology.orchestra.run.smithi012.stdout:559.out 2023-07-08T12:40:56.646 INFO:teuthology.orchestra.run.smithi012.stdout:560 2023-07-08T12:40:56.647 INFO:teuthology.orchestra.run.smithi012.stdout:560.out 2023-07-08T12:40:56.647 INFO:teuthology.orchestra.run.smithi012.stdout:561 2023-07-08T12:40:56.647 INFO:teuthology.orchestra.run.smithi012.stdout:561.out 2023-07-08T12:40:56.647 INFO:teuthology.orchestra.run.smithi012.stdout:562 2023-07-08T12:40:56.647 INFO:teuthology.orchestra.run.smithi012.stdout:562.out 2023-07-08T12:40:56.647 INFO:teuthology.orchestra.run.smithi012.stdout:563 2023-07-08T12:40:56.647 INFO:teuthology.orchestra.run.smithi012.stdout:563.out 2023-07-08T12:40:56.647 INFO:teuthology.orchestra.run.smithi012.stdout:564 2023-07-08T12:40:56.648 INFO:teuthology.orchestra.run.smithi012.stdout:564.out 2023-07-08T12:40:56.648 INFO:teuthology.orchestra.run.smithi012.stdout:565 2023-07-08T12:40:56.648 INFO:teuthology.orchestra.run.smithi012.stdout:565.out 2023-07-08T12:40:56.648 INFO:teuthology.orchestra.run.smithi012.stdout:566 2023-07-08T12:40:56.648 INFO:teuthology.orchestra.run.smithi012.stdout:566.out 2023-07-08T12:40:56.648 INFO:teuthology.orchestra.run.smithi012.stdout:567 2023-07-08T12:40:56.648 INFO:teuthology.orchestra.run.smithi012.stdout:567.out 2023-07-08T12:40:56.648 INFO:teuthology.orchestra.run.smithi012.stdout:568 2023-07-08T12:40:56.649 INFO:teuthology.orchestra.run.smithi012.stdout:568.out 2023-07-08T12:40:56.649 INFO:teuthology.orchestra.run.smithi012.stdout:569 2023-07-08T12:40:56.649 INFO:teuthology.orchestra.run.smithi012.stdout:569.out 2023-07-08T12:40:56.649 INFO:teuthology.orchestra.run.smithi012.stdout:570 2023-07-08T12:40:56.649 INFO:teuthology.orchestra.run.smithi012.stdout:570.out 2023-07-08T12:40:56.649 INFO:teuthology.orchestra.run.smithi012.stdout:571 2023-07-08T12:40:56.649 INFO:teuthology.orchestra.run.smithi012.stdout:571.out 2023-07-08T12:40:56.649 INFO:teuthology.orchestra.run.smithi012.stdout:572 2023-07-08T12:40:56.650 INFO:teuthology.orchestra.run.smithi012.stdout:572.out 2023-07-08T12:40:56.650 INFO:teuthology.orchestra.run.smithi012.stdout:573 2023-07-08T12:40:56.650 INFO:teuthology.orchestra.run.smithi012.stdout:573.out 2023-07-08T12:40:56.650 INFO:teuthology.orchestra.run.smithi012.stdout:574 2023-07-08T12:40:56.650 INFO:teuthology.orchestra.run.smithi012.stdout:574.out 2023-07-08T12:40:56.650 INFO:teuthology.orchestra.run.smithi012.stdout:575 2023-07-08T12:40:56.650 INFO:teuthology.orchestra.run.smithi012.stdout:575.out 2023-07-08T12:40:56.650 INFO:teuthology.orchestra.run.smithi012.stdout:576 2023-07-08T12:40:56.651 INFO:teuthology.orchestra.run.smithi012.stdout:576.out 2023-07-08T12:40:56.651 INFO:teuthology.orchestra.run.smithi012.stdout:577 2023-07-08T12:40:56.651 INFO:teuthology.orchestra.run.smithi012.stdout:577.out 2023-07-08T12:40:56.651 INFO:teuthology.orchestra.run.smithi012.stdout:578 2023-07-08T12:40:56.651 INFO:teuthology.orchestra.run.smithi012.stdout:578.out 2023-07-08T12:40:56.651 INFO:teuthology.orchestra.run.smithi012.stdout:579 2023-07-08T12:40:56.651 INFO:teuthology.orchestra.run.smithi012.stdout:579.out 2023-07-08T12:40:56.651 INFO:teuthology.orchestra.run.smithi012.stdout:580 2023-07-08T12:40:56.652 INFO:teuthology.orchestra.run.smithi012.stdout:580.out 2023-07-08T12:40:56.652 INFO:teuthology.orchestra.run.smithi012.stdout:581 2023-07-08T12:40:56.652 INFO:teuthology.orchestra.run.smithi012.stdout:581.out 2023-07-08T12:40:56.652 INFO:teuthology.orchestra.run.smithi012.stdout:582 2023-07-08T12:40:56.652 INFO:teuthology.orchestra.run.smithi012.stdout:582.out 2023-07-08T12:40:56.652 INFO:teuthology.orchestra.run.smithi012.stdout:583 2023-07-08T12:40:56.652 INFO:teuthology.orchestra.run.smithi012.stdout:583.out 2023-07-08T12:40:56.652 INFO:teuthology.orchestra.run.smithi012.stdout:584 2023-07-08T12:40:56.653 INFO:teuthology.orchestra.run.smithi012.stdout:584.out 2023-07-08T12:40:56.653 INFO:teuthology.orchestra.run.smithi012.stdout:585 2023-07-08T12:40:56.653 INFO:teuthology.orchestra.run.smithi012.stdout:585.out 2023-07-08T12:40:56.653 INFO:teuthology.orchestra.run.smithi012.stdout:586 2023-07-08T12:40:56.653 INFO:teuthology.orchestra.run.smithi012.stdout:586.out 2023-07-08T12:40:56.653 INFO:teuthology.orchestra.run.smithi012.stdout:587 2023-07-08T12:40:56.653 INFO:teuthology.orchestra.run.smithi012.stdout:587.out 2023-07-08T12:40:56.653 INFO:teuthology.orchestra.run.smithi012.stdout:588 2023-07-08T12:40:56.654 INFO:teuthology.orchestra.run.smithi012.stdout:588.out 2023-07-08T12:40:56.654 INFO:teuthology.orchestra.run.smithi012.stdout:589 2023-07-08T12:40:56.654 INFO:teuthology.orchestra.run.smithi012.stdout:589.out 2023-07-08T12:40:56.654 INFO:teuthology.orchestra.run.smithi012.stdout:590 2023-07-08T12:40:56.654 INFO:teuthology.orchestra.run.smithi012.stdout:590.out 2023-07-08T12:40:56.654 INFO:teuthology.orchestra.run.smithi012.stdout:591 2023-07-08T12:40:56.654 INFO:teuthology.orchestra.run.smithi012.stdout:591.out 2023-07-08T12:40:56.654 INFO:teuthology.orchestra.run.smithi012.stdout:592 2023-07-08T12:40:56.655 INFO:teuthology.orchestra.run.smithi012.stdout:592.out 2023-07-08T12:40:56.655 INFO:teuthology.orchestra.run.smithi012.stdout:593 2023-07-08T12:40:56.655 INFO:teuthology.orchestra.run.smithi012.stdout:593.out 2023-07-08T12:40:56.655 INFO:teuthology.orchestra.run.smithi012.stdout:594 2023-07-08T12:40:56.655 INFO:teuthology.orchestra.run.smithi012.stdout:594.out 2023-07-08T12:40:56.655 INFO:teuthology.orchestra.run.smithi012.stdout:595 2023-07-08T12:40:56.655 INFO:teuthology.orchestra.run.smithi012.stdout:595.out 2023-07-08T12:40:56.655 INFO:teuthology.orchestra.run.smithi012.stdout:596 2023-07-08T12:40:56.656 INFO:teuthology.orchestra.run.smithi012.stdout:596.out 2023-07-08T12:40:56.656 INFO:teuthology.orchestra.run.smithi012.stdout:597 2023-07-08T12:40:56.656 INFO:teuthology.orchestra.run.smithi012.stdout:597.out 2023-07-08T12:40:56.656 INFO:teuthology.orchestra.run.smithi012.stdout:598 2023-07-08T12:40:56.656 INFO:teuthology.orchestra.run.smithi012.stdout:598.out 2023-07-08T12:40:56.656 INFO:teuthology.orchestra.run.smithi012.stdout:599 2023-07-08T12:40:56.656 INFO:teuthology.orchestra.run.smithi012.stdout:599.out 2023-07-08T12:40:56.656 INFO:teuthology.orchestra.run.smithi012.stdout:600 2023-07-08T12:40:56.657 INFO:teuthology.orchestra.run.smithi012.stdout:600.out 2023-07-08T12:40:56.657 INFO:teuthology.orchestra.run.smithi012.stdout:601 2023-07-08T12:40:56.657 INFO:teuthology.orchestra.run.smithi012.stdout:601.out 2023-07-08T12:40:56.657 INFO:teuthology.orchestra.run.smithi012.stdout:602 2023-07-08T12:40:56.657 INFO:teuthology.orchestra.run.smithi012.stdout:602.out 2023-07-08T12:40:56.657 INFO:teuthology.orchestra.run.smithi012.stdout:603 2023-07-08T12:40:56.657 INFO:teuthology.orchestra.run.smithi012.stdout:603.out 2023-07-08T12:40:56.657 INFO:teuthology.orchestra.run.smithi012.stdout:604 2023-07-08T12:40:56.658 INFO:teuthology.orchestra.run.smithi012.stdout:604.out 2023-07-08T12:40:56.658 INFO:teuthology.orchestra.run.smithi012.stdout:605 2023-07-08T12:40:56.658 INFO:teuthology.orchestra.run.smithi012.stdout:605.out 2023-07-08T12:40:56.658 INFO:teuthology.orchestra.run.smithi012.stdout:606 2023-07-08T12:40:56.658 INFO:teuthology.orchestra.run.smithi012.stdout:606.out 2023-07-08T12:40:56.658 INFO:teuthology.orchestra.run.smithi012.stdout:607 2023-07-08T12:40:56.658 INFO:teuthology.orchestra.run.smithi012.stdout:607.out 2023-07-08T12:40:56.658 INFO:teuthology.orchestra.run.smithi012.stdout:608 2023-07-08T12:40:56.659 INFO:teuthology.orchestra.run.smithi012.stdout:608.out 2023-07-08T12:40:56.659 INFO:teuthology.orchestra.run.smithi012.stdout:609 2023-07-08T12:40:56.659 INFO:teuthology.orchestra.run.smithi012.stdout:609.out 2023-07-08T12:40:56.659 INFO:teuthology.orchestra.run.smithi012.stdout:610 2023-07-08T12:40:56.659 INFO:teuthology.orchestra.run.smithi012.stdout:610.out 2023-07-08T12:40:56.659 INFO:teuthology.orchestra.run.smithi012.stdout:611 2023-07-08T12:40:56.659 INFO:teuthology.orchestra.run.smithi012.stdout:611.out 2023-07-08T12:40:56.659 INFO:teuthology.orchestra.run.smithi012.stdout:612 2023-07-08T12:40:56.660 INFO:teuthology.orchestra.run.smithi012.stdout:612.out 2023-07-08T12:40:56.660 INFO:teuthology.orchestra.run.smithi012.stdout:613 2023-07-08T12:40:56.660 INFO:teuthology.orchestra.run.smithi012.stdout:613.out 2023-07-08T12:40:56.660 INFO:teuthology.orchestra.run.smithi012.stdout:614 2023-07-08T12:40:56.660 INFO:teuthology.orchestra.run.smithi012.stdout:614.out 2023-07-08T12:40:56.660 INFO:teuthology.orchestra.run.smithi012.stdout:615 2023-07-08T12:40:56.660 INFO:teuthology.orchestra.run.smithi012.stdout:615.out 2023-07-08T12:40:56.660 INFO:teuthology.orchestra.run.smithi012.stdout:616 2023-07-08T12:40:56.661 INFO:teuthology.orchestra.run.smithi012.stdout:616.out 2023-07-08T12:40:56.661 INFO:teuthology.orchestra.run.smithi012.stdout:617 2023-07-08T12:40:56.661 INFO:teuthology.orchestra.run.smithi012.stdout:617.out 2023-07-08T12:40:56.661 INFO:teuthology.orchestra.run.smithi012.stdout:618 2023-07-08T12:40:56.661 INFO:teuthology.orchestra.run.smithi012.stdout:618.out 2023-07-08T12:40:56.661 INFO:teuthology.orchestra.run.smithi012.stdout:619 2023-07-08T12:40:56.661 INFO:teuthology.orchestra.run.smithi012.stdout:619.out 2023-07-08T12:40:56.661 INFO:teuthology.orchestra.run.smithi012.stdout:620 2023-07-08T12:40:56.662 INFO:teuthology.orchestra.run.smithi012.stdout:620.out 2023-07-08T12:40:56.662 INFO:teuthology.orchestra.run.smithi012.stdout:621 2023-07-08T12:40:56.662 INFO:teuthology.orchestra.run.smithi012.stdout:621.out 2023-07-08T12:40:56.662 INFO:teuthology.orchestra.run.smithi012.stdout:622 2023-07-08T12:40:56.662 INFO:teuthology.orchestra.run.smithi012.stdout:622.out 2023-07-08T12:40:56.662 INFO:teuthology.orchestra.run.smithi012.stdout:623 2023-07-08T12:40:56.662 INFO:teuthology.orchestra.run.smithi012.stdout:623.out 2023-07-08T12:40:56.662 INFO:teuthology.orchestra.run.smithi012.stdout:624 2023-07-08T12:40:56.663 INFO:teuthology.orchestra.run.smithi012.stdout:624.out 2023-07-08T12:40:56.663 INFO:teuthology.orchestra.run.smithi012.stdout:625 2023-07-08T12:40:56.663 INFO:teuthology.orchestra.run.smithi012.stdout:625.out 2023-07-08T12:40:56.663 INFO:teuthology.orchestra.run.smithi012.stdout:626 2023-07-08T12:40:56.663 INFO:teuthology.orchestra.run.smithi012.stdout:626.out 2023-07-08T12:40:56.663 INFO:teuthology.orchestra.run.smithi012.stdout:627 2023-07-08T12:40:56.663 INFO:teuthology.orchestra.run.smithi012.stdout:627.out 2023-07-08T12:40:56.663 INFO:teuthology.orchestra.run.smithi012.stdout:628 2023-07-08T12:40:56.664 INFO:teuthology.orchestra.run.smithi012.stdout:628.out 2023-07-08T12:40:56.664 INFO:teuthology.orchestra.run.smithi012.stdout:629 2023-07-08T12:40:56.664 INFO:teuthology.orchestra.run.smithi012.stdout:629.out 2023-07-08T12:40:56.664 INFO:teuthology.orchestra.run.smithi012.stdout:630 2023-07-08T12:40:56.664 INFO:teuthology.orchestra.run.smithi012.stdout:630.out 2023-07-08T12:40:56.664 INFO:teuthology.orchestra.run.smithi012.stdout:631 2023-07-08T12:40:56.664 INFO:teuthology.orchestra.run.smithi012.stdout:631.out 2023-07-08T12:40:56.664 INFO:teuthology.orchestra.run.smithi012.stdout:632 2023-07-08T12:40:56.665 INFO:teuthology.orchestra.run.smithi012.stdout:632.out 2023-07-08T12:40:56.665 INFO:teuthology.orchestra.run.smithi012.stdout:633 2023-07-08T12:40:56.665 INFO:teuthology.orchestra.run.smithi012.stdout:633.out 2023-07-08T12:40:56.665 INFO:teuthology.orchestra.run.smithi012.stdout:634 2023-07-08T12:40:56.665 INFO:teuthology.orchestra.run.smithi012.stdout:634.out 2023-07-08T12:40:56.665 INFO:teuthology.orchestra.run.smithi012.stdout:635 2023-07-08T12:40:56.665 INFO:teuthology.orchestra.run.smithi012.stdout:635.out 2023-07-08T12:40:56.665 INFO:teuthology.orchestra.run.smithi012.stdout:636 2023-07-08T12:40:56.666 INFO:teuthology.orchestra.run.smithi012.stdout:636.out 2023-07-08T12:40:56.666 INFO:teuthology.orchestra.run.smithi012.stdout:637 2023-07-08T12:40:56.666 INFO:teuthology.orchestra.run.smithi012.stdout:637.out 2023-07-08T12:40:56.666 INFO:teuthology.orchestra.run.smithi012.stdout:638 2023-07-08T12:40:56.666 INFO:teuthology.orchestra.run.smithi012.stdout:638.out 2023-07-08T12:40:56.666 INFO:teuthology.orchestra.run.smithi012.stdout:639 2023-07-08T12:40:56.666 INFO:teuthology.orchestra.run.smithi012.stdout:639.out 2023-07-08T12:40:56.666 INFO:teuthology.orchestra.run.smithi012.stdout:640 2023-07-08T12:40:56.667 INFO:teuthology.orchestra.run.smithi012.stdout:640.out 2023-07-08T12:40:56.667 INFO:teuthology.orchestra.run.smithi012.stdout:641 2023-07-08T12:40:56.667 INFO:teuthology.orchestra.run.smithi012.stdout:641.out 2023-07-08T12:40:56.667 INFO:teuthology.orchestra.run.smithi012.stdout:642 2023-07-08T12:40:56.667 INFO:teuthology.orchestra.run.smithi012.stdout:642.out 2023-07-08T12:40:56.667 INFO:teuthology.orchestra.run.smithi012.stdout:643 2023-07-08T12:40:56.667 INFO:teuthology.orchestra.run.smithi012.stdout:643.out 2023-07-08T12:40:56.667 INFO:teuthology.orchestra.run.smithi012.stdout:644 2023-07-08T12:40:56.668 INFO:teuthology.orchestra.run.smithi012.stdout:644.out 2023-07-08T12:40:56.668 INFO:teuthology.orchestra.run.smithi012.stdout:645 2023-07-08T12:40:56.668 INFO:teuthology.orchestra.run.smithi012.stdout:645.out 2023-07-08T12:40:56.668 INFO:teuthology.orchestra.run.smithi012.stdout:646 2023-07-08T12:40:56.668 INFO:teuthology.orchestra.run.smithi012.stdout:646.out 2023-07-08T12:40:56.668 INFO:teuthology.orchestra.run.smithi012.stdout:647 2023-07-08T12:40:56.668 INFO:teuthology.orchestra.run.smithi012.stdout:647.out 2023-07-08T12:40:56.668 INFO:teuthology.orchestra.run.smithi012.stdout:648 2023-07-08T12:40:56.669 INFO:teuthology.orchestra.run.smithi012.stdout:648.out 2023-07-08T12:40:56.669 INFO:teuthology.orchestra.run.smithi012.stdout:649 2023-07-08T12:40:56.669 INFO:teuthology.orchestra.run.smithi012.stdout:649.out 2023-07-08T12:40:56.669 INFO:teuthology.orchestra.run.smithi012.stdout:650 2023-07-08T12:40:56.669 INFO:teuthology.orchestra.run.smithi012.stdout:650.out 2023-07-08T12:40:56.669 INFO:teuthology.orchestra.run.smithi012.stdout:651 2023-07-08T12:40:56.669 INFO:teuthology.orchestra.run.smithi012.stdout:651.out 2023-07-08T12:40:56.669 INFO:teuthology.orchestra.run.smithi012.stdout:652 2023-07-08T12:40:56.670 INFO:teuthology.orchestra.run.smithi012.stdout:652.out 2023-07-08T12:40:56.670 INFO:teuthology.orchestra.run.smithi012.stdout:653 2023-07-08T12:40:56.670 INFO:teuthology.orchestra.run.smithi012.stdout:653.out 2023-07-08T12:40:56.670 INFO:teuthology.orchestra.run.smithi012.stdout:654 2023-07-08T12:40:56.670 INFO:teuthology.orchestra.run.smithi012.stdout:654.out 2023-07-08T12:40:56.670 INFO:teuthology.orchestra.run.smithi012.stdout:655 2023-07-08T12:40:56.670 INFO:teuthology.orchestra.run.smithi012.stdout:655.out 2023-07-08T12:40:56.670 INFO:teuthology.orchestra.run.smithi012.stdout:656 2023-07-08T12:40:56.671 INFO:teuthology.orchestra.run.smithi012.stdout:656.out 2023-07-08T12:40:56.671 INFO:teuthology.orchestra.run.smithi012.stdout:657 2023-07-08T12:40:56.671 INFO:teuthology.orchestra.run.smithi012.stdout:657.out 2023-07-08T12:40:56.671 INFO:teuthology.orchestra.run.smithi012.stdout:658 2023-07-08T12:40:56.671 INFO:teuthology.orchestra.run.smithi012.stdout:658.out 2023-07-08T12:40:56.671 INFO:teuthology.orchestra.run.smithi012.stdout:659 2023-07-08T12:40:56.671 INFO:teuthology.orchestra.run.smithi012.stdout:659.out 2023-07-08T12:40:56.671 INFO:teuthology.orchestra.run.smithi012.stdout:660 2023-07-08T12:40:56.671 INFO:teuthology.orchestra.run.smithi012.stdout:660.out 2023-07-08T12:40:56.672 INFO:teuthology.orchestra.run.smithi012.stdout:661 2023-07-08T12:40:56.672 INFO:teuthology.orchestra.run.smithi012.stdout:661.out 2023-07-08T12:40:56.672 INFO:teuthology.orchestra.run.smithi012.stdout:662 2023-07-08T12:40:56.672 INFO:teuthology.orchestra.run.smithi012.stdout:662.out 2023-07-08T12:40:56.672 INFO:teuthology.orchestra.run.smithi012.stdout:663 2023-07-08T12:40:56.672 INFO:teuthology.orchestra.run.smithi012.stdout:663.out 2023-07-08T12:40:56.672 INFO:teuthology.orchestra.run.smithi012.stdout:664 2023-07-08T12:40:56.673 INFO:teuthology.orchestra.run.smithi012.stdout:664.out 2023-07-08T12:40:56.673 INFO:teuthology.orchestra.run.smithi012.stdout:665 2023-07-08T12:40:56.673 INFO:teuthology.orchestra.run.smithi012.stdout:665.out 2023-07-08T12:40:56.673 INFO:teuthology.orchestra.run.smithi012.stdout:666 2023-07-08T12:40:56.673 INFO:teuthology.orchestra.run.smithi012.stdout:666.out 2023-07-08T12:40:56.673 INFO:teuthology.orchestra.run.smithi012.stdout:667 2023-07-08T12:40:56.673 INFO:teuthology.orchestra.run.smithi012.stdout:667.out 2023-07-08T12:40:56.673 INFO:teuthology.orchestra.run.smithi012.stdout:668 2023-07-08T12:40:56.674 INFO:teuthology.orchestra.run.smithi012.stdout:668.out 2023-07-08T12:40:56.674 INFO:teuthology.orchestra.run.smithi012.stdout:669 2023-07-08T12:40:56.674 INFO:teuthology.orchestra.run.smithi012.stdout:669.out 2023-07-08T12:40:56.674 INFO:teuthology.orchestra.run.smithi012.stdout:670 2023-07-08T12:40:56.674 INFO:teuthology.orchestra.run.smithi012.stdout:670.out 2023-07-08T12:40:56.674 INFO:teuthology.orchestra.run.smithi012.stdout:671 2023-07-08T12:40:56.674 INFO:teuthology.orchestra.run.smithi012.stdout:671.out 2023-07-08T12:40:56.674 INFO:teuthology.orchestra.run.smithi012.stdout:672 2023-07-08T12:40:56.674 INFO:teuthology.orchestra.run.smithi012.stdout:672.out 2023-07-08T12:40:56.675 INFO:teuthology.orchestra.run.smithi012.stdout:673 2023-07-08T12:40:56.675 INFO:teuthology.orchestra.run.smithi012.stdout:673.out 2023-07-08T12:40:56.675 INFO:teuthology.orchestra.run.smithi012.stdout:674 2023-07-08T12:40:56.675 INFO:teuthology.orchestra.run.smithi012.stdout:674.out 2023-07-08T12:40:56.675 INFO:teuthology.orchestra.run.smithi012.stdout:675 2023-07-08T12:40:56.675 INFO:teuthology.orchestra.run.smithi012.stdout:675.out 2023-07-08T12:40:56.675 INFO:teuthology.orchestra.run.smithi012.stdout:676 2023-07-08T12:40:56.675 INFO:teuthology.orchestra.run.smithi012.stdout:676.out 2023-07-08T12:40:56.676 INFO:teuthology.orchestra.run.smithi012.stdout:677 2023-07-08T12:40:56.676 INFO:teuthology.orchestra.run.smithi012.stdout:677.out 2023-07-08T12:40:56.676 INFO:teuthology.orchestra.run.smithi012.stdout:678 2023-07-08T12:40:56.676 INFO:teuthology.orchestra.run.smithi012.stdout:678.out 2023-07-08T12:40:56.676 INFO:teuthology.orchestra.run.smithi012.stdout:679 2023-07-08T12:40:56.676 INFO:teuthology.orchestra.run.smithi012.stdout:679.out 2023-07-08T12:40:56.676 INFO:teuthology.orchestra.run.smithi012.stdout:680 2023-07-08T12:40:56.677 INFO:teuthology.orchestra.run.smithi012.stdout:680.out 2023-07-08T12:40:56.677 INFO:teuthology.orchestra.run.smithi012.stdout:681 2023-07-08T12:40:56.677 INFO:teuthology.orchestra.run.smithi012.stdout:681.out 2023-07-08T12:40:56.677 INFO:teuthology.orchestra.run.smithi012.stdout:682 2023-07-08T12:40:56.677 INFO:teuthology.orchestra.run.smithi012.stdout:682.out 2023-07-08T12:40:56.677 INFO:teuthology.orchestra.run.smithi012.stdout:683 2023-07-08T12:40:56.677 INFO:teuthology.orchestra.run.smithi012.stdout:683.out 2023-07-08T12:40:56.677 INFO:teuthology.orchestra.run.smithi012.stdout:684 2023-07-08T12:40:56.678 INFO:teuthology.orchestra.run.smithi012.stdout:684.out 2023-07-08T12:40:56.678 INFO:teuthology.orchestra.run.smithi012.stdout:685 2023-07-08T12:40:56.678 INFO:teuthology.orchestra.run.smithi012.stdout:685.out 2023-07-08T12:40:56.678 INFO:teuthology.orchestra.run.smithi012.stdout:686 2023-07-08T12:40:56.678 INFO:teuthology.orchestra.run.smithi012.stdout:686.out 2023-07-08T12:40:56.678 INFO:teuthology.orchestra.run.smithi012.stdout:687 2023-07-08T12:40:56.678 INFO:teuthology.orchestra.run.smithi012.stdout:687.out 2023-07-08T12:40:56.678 INFO:teuthology.orchestra.run.smithi012.stdout:688 2023-07-08T12:40:56.679 INFO:teuthology.orchestra.run.smithi012.stdout:688.out 2023-07-08T12:40:56.679 INFO:teuthology.orchestra.run.smithi012.stdout:689 2023-07-08T12:40:56.679 INFO:teuthology.orchestra.run.smithi012.stdout:689.out 2023-07-08T12:40:56.679 INFO:teuthology.orchestra.run.smithi012.stdout:690 2023-07-08T12:40:56.679 INFO:teuthology.orchestra.run.smithi012.stdout:690.out 2023-07-08T12:40:56.679 INFO:teuthology.orchestra.run.smithi012.stdout:691 2023-07-08T12:40:56.679 INFO:teuthology.orchestra.run.smithi012.stdout:691.out 2023-07-08T12:40:56.679 INFO:teuthology.orchestra.run.smithi012.stdout:692 2023-07-08T12:40:56.680 INFO:teuthology.orchestra.run.smithi012.stdout:692.out 2023-07-08T12:40:56.680 INFO:teuthology.orchestra.run.smithi012.stdout:693 2023-07-08T12:40:56.680 INFO:teuthology.orchestra.run.smithi012.stdout:693.out 2023-07-08T12:40:56.680 INFO:teuthology.orchestra.run.smithi012.stdout:694 2023-07-08T12:40:56.680 INFO:teuthology.orchestra.run.smithi012.stdout:694.out 2023-07-08T12:40:56.680 INFO:teuthology.orchestra.run.smithi012.stdout:695 2023-07-08T12:40:56.680 INFO:teuthology.orchestra.run.smithi012.stdout:695.out 2023-07-08T12:40:56.680 INFO:teuthology.orchestra.run.smithi012.stdout:696 2023-07-08T12:40:56.681 INFO:teuthology.orchestra.run.smithi012.stdout:696.out 2023-07-08T12:40:56.681 INFO:teuthology.orchestra.run.smithi012.stdout:697 2023-07-08T12:40:56.681 INFO:teuthology.orchestra.run.smithi012.stdout:697.out 2023-07-08T12:40:56.681 INFO:teuthology.orchestra.run.smithi012.stdout:698 2023-07-08T12:40:56.681 INFO:teuthology.orchestra.run.smithi012.stdout:698.out 2023-07-08T12:40:56.681 INFO:teuthology.orchestra.run.smithi012.stdout:699 2023-07-08T12:40:56.681 INFO:teuthology.orchestra.run.smithi012.stdout:699.out 2023-07-08T12:40:56.681 INFO:teuthology.orchestra.run.smithi012.stdout:700 2023-07-08T12:40:56.682 INFO:teuthology.orchestra.run.smithi012.stdout:700.out 2023-07-08T12:40:56.682 INFO:teuthology.orchestra.run.smithi012.stdout:701 2023-07-08T12:40:56.682 INFO:teuthology.orchestra.run.smithi012.stdout:701.out 2023-07-08T12:40:56.682 INFO:teuthology.orchestra.run.smithi012.stdout:702 2023-07-08T12:40:56.682 INFO:teuthology.orchestra.run.smithi012.stdout:702.out 2023-07-08T12:40:56.682 INFO:teuthology.orchestra.run.smithi012.stdout:703 2023-07-08T12:40:56.682 INFO:teuthology.orchestra.run.smithi012.stdout:703.out 2023-07-08T12:40:56.682 INFO:teuthology.orchestra.run.smithi012.stdout:704 2023-07-08T12:40:56.683 INFO:teuthology.orchestra.run.smithi012.stdout:704.out 2023-07-08T12:40:56.683 INFO:teuthology.orchestra.run.smithi012.stdout:705 2023-07-08T12:40:56.683 INFO:teuthology.orchestra.run.smithi012.stdout:705.out 2023-07-08T12:40:56.683 INFO:teuthology.orchestra.run.smithi012.stdout:706 2023-07-08T12:40:56.683 INFO:teuthology.orchestra.run.smithi012.stdout:706.out 2023-07-08T12:40:56.683 INFO:teuthology.orchestra.run.smithi012.stdout:707 2023-07-08T12:40:56.683 INFO:teuthology.orchestra.run.smithi012.stdout:707.out 2023-07-08T12:40:56.683 INFO:teuthology.orchestra.run.smithi012.stdout:708 2023-07-08T12:40:56.684 INFO:teuthology.orchestra.run.smithi012.stdout:708.out 2023-07-08T12:40:56.684 INFO:teuthology.orchestra.run.smithi012.stdout:709 2023-07-08T12:40:56.684 INFO:teuthology.orchestra.run.smithi012.stdout:709.out 2023-07-08T12:40:56.684 INFO:teuthology.orchestra.run.smithi012.stdout:710 2023-07-08T12:40:56.684 INFO:teuthology.orchestra.run.smithi012.stdout:710.out 2023-07-08T12:40:56.684 INFO:teuthology.orchestra.run.smithi012.stdout:711 2023-07-08T12:40:56.684 INFO:teuthology.orchestra.run.smithi012.stdout:711.out 2023-07-08T12:40:56.684 INFO:teuthology.orchestra.run.smithi012.stdout:712 2023-07-08T12:40:56.685 INFO:teuthology.orchestra.run.smithi012.stdout:712.out 2023-07-08T12:40:56.685 INFO:teuthology.orchestra.run.smithi012.stdout:713 2023-07-08T12:40:56.685 INFO:teuthology.orchestra.run.smithi012.stdout:713.out 2023-07-08T12:40:56.685 INFO:teuthology.orchestra.run.smithi012.stdout:714 2023-07-08T12:40:56.685 INFO:teuthology.orchestra.run.smithi012.stdout:714.out 2023-07-08T12:40:56.685 INFO:teuthology.orchestra.run.smithi012.stdout:715 2023-07-08T12:40:56.685 INFO:teuthology.orchestra.run.smithi012.stdout:715.out 2023-07-08T12:40:56.685 INFO:teuthology.orchestra.run.smithi012.stdout:716 2023-07-08T12:40:56.686 INFO:teuthology.orchestra.run.smithi012.stdout:716.out 2023-07-08T12:40:56.686 INFO:teuthology.orchestra.run.smithi012.stdout:717 2023-07-08T12:40:56.686 INFO:teuthology.orchestra.run.smithi012.stdout:717.out 2023-07-08T12:40:56.686 INFO:teuthology.orchestra.run.smithi012.stdout:718 2023-07-08T12:40:56.686 INFO:teuthology.orchestra.run.smithi012.stdout:718.out 2023-07-08T12:40:56.686 INFO:teuthology.orchestra.run.smithi012.stdout:719 2023-07-08T12:40:56.686 INFO:teuthology.orchestra.run.smithi012.stdout:719.out 2023-07-08T12:40:56.686 INFO:teuthology.orchestra.run.smithi012.stdout:720 2023-07-08T12:40:56.687 INFO:teuthology.orchestra.run.smithi012.stdout:720.out 2023-07-08T12:40:56.687 INFO:teuthology.orchestra.run.smithi012.stdout:721 2023-07-08T12:40:56.687 INFO:teuthology.orchestra.run.smithi012.stdout:721.out 2023-07-08T12:40:56.687 INFO:teuthology.orchestra.run.smithi012.stdout:722 2023-07-08T12:40:56.687 INFO:teuthology.orchestra.run.smithi012.stdout:722.out 2023-07-08T12:40:56.687 INFO:teuthology.orchestra.run.smithi012.stdout:723 2023-07-08T12:40:56.687 INFO:teuthology.orchestra.run.smithi012.stdout:723.out 2023-07-08T12:40:56.687 INFO:teuthology.orchestra.run.smithi012.stdout:724 2023-07-08T12:40:56.688 INFO:teuthology.orchestra.run.smithi012.stdout:724.out 2023-07-08T12:40:56.688 INFO:teuthology.orchestra.run.smithi012.stdout:725 2023-07-08T12:40:56.688 INFO:teuthology.orchestra.run.smithi012.stdout:725.out 2023-07-08T12:40:56.688 INFO:teuthology.orchestra.run.smithi012.stdout:726 2023-07-08T12:40:56.688 INFO:teuthology.orchestra.run.smithi012.stdout:726.out 2023-07-08T12:40:56.688 INFO:teuthology.orchestra.run.smithi012.stdout:727 2023-07-08T12:40:56.688 INFO:teuthology.orchestra.run.smithi012.stdout:727.out 2023-07-08T12:40:56.688 INFO:teuthology.orchestra.run.smithi012.stdout:728 2023-07-08T12:40:56.689 INFO:teuthology.orchestra.run.smithi012.stdout:728.out 2023-07-08T12:40:56.689 INFO:teuthology.orchestra.run.smithi012.stdout:group.list 2023-07-08T12:40:56.689 INFO:teuthology.orchestra.run.smithi012.stdout:Makefile 2023-07-08T12:40:56.689 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2023-07-08T12:40:56.929 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:40:56.933 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:40:56.934 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:40:56.934 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:40:56.935 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:41:01.595 INFO:teuthology.orchestra.run.smithi012.stdout:generic/001 4s 2023-07-08T12:41:01.600 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/001 2023-07-08T12:41:01.600 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:41:01.602 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:41:02.356 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:41:02.357 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2023-07-08T12:41:02.580 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:41:02.584 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:41:02.584 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:41:02.585 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:41:02.585 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:41:03.112 INFO:teuthology.orchestra.run.smithi012.stdout:generic/002 1s 2023-07-08T12:41:03.124 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/002 2023-07-08T12:41:03.124 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:41:03.130 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:41:03.297 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:41:03.298 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2023-07-08T12:41:03.524 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:41:03.528 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:41:03.528 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:41:03.529 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:41:03.529 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:41:03.868 INFO:teuthology.orchestra.run.smithi012.stdout:generic/003 [not run] atime not maintained by ceph 2023-07-08T12:41:03.872 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/003 2023-07-08T12:41:03.872 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/003 2023-07-08T12:41:03.872 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:41:03.874 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:41:03.895 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:41:03.895 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2023-07-08T12:41:04.119 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:41:04.123 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:41:04.123 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:41:04.124 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:41:04.124 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:41:04.470 INFO:teuthology.orchestra.run.smithi012.stdout:generic/004 [not run] O_TMPFILE is not supported 2023-07-08T12:41:04.474 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/004 2023-07-08T12:41:04.474 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/004 2023-07-08T12:41:04.474 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:41:04.476 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:41:04.503 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:41:04.503 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2023-07-08T12:41:04.722 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:41:04.726 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:41:04.727 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:41:04.728 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:41:04.728 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:41:05.200 INFO:teuthology.orchestra.run.smithi012.stdout:generic/005 1s 2023-07-08T12:41:05.205 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/005 2023-07-08T12:41:05.205 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:41:05.207 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:41:05.297 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:41:05.298 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2023-07-08T12:41:05.528 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:41:05.531 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:41:05.532 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:41:05.533 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:41:05.533 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:41:06.411 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:41:06.414 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:41:30.643 INFO:teuthology.orchestra.run.smithi012.stdout:generic/006 25s 2023-07-08T12:41:30.652 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/006 2023-07-08T12:41:30.653 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:41:30.654 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:41:30.736 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:41:30.737 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2023-07-08T12:41:30.965 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:41:30.969 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:41:30.970 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:41:30.971 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:41:30.971 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:41:36.658 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:41:36.660 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:42:06.733 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:42:06.735 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:42:20.745 INFO:teuthology.orchestra.run.smithi012.stdout:generic/007 49s 2023-07-08T12:42:20.750 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/007 2023-07-08T12:42:20.750 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:42:20.752 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:42:20.837 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:42:20.838 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2023-07-08T12:42:21.202 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:42:21.206 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:42:21.206 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:42:21.207 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:42:21.207 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:42:21.613 INFO:teuthology.orchestra.run.smithi012.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-07-08T12:42:21.617 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/008 2023-07-08T12:42:21.617 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/008 2023-07-08T12:42:21.617 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:42:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:42:21.651 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:42:21.651 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2023-07-08T12:42:21.862 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:42:21.867 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:42:21.867 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:42:21.868 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:42:21.868 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:42:22.264 INFO:teuthology.orchestra.run.smithi012.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-07-08T12:42:22.268 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/009 2023-07-08T12:42:22.268 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/009 2023-07-08T12:42:22.268 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:42:22.270 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:42:22.297 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:42:22.298 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2023-07-08T12:42:22.505 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:42:22.510 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:42:22.511 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:42:22.511 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:42:22.512 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:42:22.813 INFO:teuthology.orchestra.run.smithi012.stdout:generic/010 [not run] /tmp/tmp.HvAMMOKiw7xfstests-dev/src/dbtest not built 2023-07-08T12:42:22.817 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/010 2023-07-08T12:42:22.818 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/010 2023-07-08T12:42:22.818 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:42:22.819 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:42:22.842 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:42:22.843 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2023-07-08T12:42:23.063 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:42:23.067 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:42:23.068 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:42:23.069 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:42:23.069 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:42:36.813 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:42:36.815 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:42:54.129 INFO:teuthology.orchestra.run.smithi012.stdout:generic/011 31s 2023-07-08T12:42:54.134 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/011 2023-07-08T12:42:54.134 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:42:54.136 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:42:54.470 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:42:54.471 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2023-07-08T12:42:54.693 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:42:54.697 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:42:54.697 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:42:54.698 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:42:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:42:55.105 INFO:teuthology.orchestra.run.smithi012.stdout:generic/012 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T12:42:55.109 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/012 2023-07-08T12:42:55.109 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/012 2023-07-08T12:42:55.109 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:42:55.111 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:42:55.166 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:42:55.167 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2023-07-08T12:42:55.489 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:42:55.493 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:42:55.494 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:42:55.495 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:42:55.495 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:43:06.877 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:43:06.880 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:43:37.024 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:43:37.028 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:44:06.638 INFO:teuthology.orchestra.run.smithi012.stdout:generic/013 71s 2023-07-08T12:44:06.645 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/013 2023-07-08T12:44:06.645 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:44:06.648 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:44:06.685 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:44:06.685 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2023-07-08T12:44:07.009 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:44:07.014 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:44:07.015 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:44:07.016 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:44:07.016 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:44:07.368 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:44:07.372 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:44:37.706 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:44:37.708 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:45:07.815 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:45:07.817 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:45:37.929 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:45:37.931 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:46:08.053 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:46:08.056 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:46:38.511 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:46:38.518 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:47:08.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:47:08.773 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:47:38.852 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:47:38.858 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:48:08.963 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:48:08.990 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:48:39.118 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:48:39.124 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:48:55.956 INFO:teuthology.orchestra.run.smithi012.stdout:generic/014 288s 2023-07-08T12:48:56.014 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/014 2023-07-08T12:48:56.015 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:48:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:48:56.060 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:48:56.061 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2023-07-08T12:48:56.444 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:48:56.449 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:48:56.450 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:48:56.451 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:48:56.451 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:48:56.956 INFO:teuthology.orchestra.run.smithi012.stdout:generic/015 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T12:48:56.966 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/015 2023-07-08T12:48:56.966 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/015 2023-07-08T12:48:56.966 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:48:56.972 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:48:57.004 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:48:57.005 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2023-07-08T12:48:57.400 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:48:57.405 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:48:57.405 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:48:57.406 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:48:57.406 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:48:58.109 INFO:teuthology.orchestra.run.smithi012.stdout:generic/016 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T12:48:58.116 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/016 2023-07-08T12:48:58.116 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/016 2023-07-08T12:48:58.116 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:48:58.123 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:48:58.419 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:48:58.419 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2023-07-08T12:48:58.787 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:48:58.792 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:48:58.792 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:48:58.793 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:48:58.793 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:48:59.310 INFO:teuthology.orchestra.run.smithi012.stdout:generic/017 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-07-08T12:48:59.315 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/017 2023-07-08T12:48:59.315 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/017 2023-07-08T12:48:59.316 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:48:59.318 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:48:59.353 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:48:59.353 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2023-07-08T12:48:59.733 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:48:59.738 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:48:59.739 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:48:59.740 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:48:59.740 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:49:00.463 INFO:teuthology.orchestra.run.smithi012.stdout:generic/018 [not run] defragmentation not supported for fstype "ceph" 2023-07-08T12:49:00.486 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/018 2023-07-08T12:49:00.486 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/018 2023-07-08T12:49:00.486 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:49:00.488 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:49:00.521 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:49:00.521 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2023-07-08T12:49:00.882 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:49:00.886 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:49:00.887 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:49:00.888 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:49:00.889 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:49:01.397 INFO:teuthology.orchestra.run.smithi012.stdout:generic/019 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:49:01.401 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/019 2023-07-08T12:49:01.402 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/019 2023-07-08T12:49:01.402 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:49:01.405 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:49:01.433 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:49:01.433 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2023-07-08T12:49:01.792 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:49:01.797 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:49:01.797 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:49:01.798 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:49:01.798 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:49:05.621 INFO:teuthology.orchestra.run.smithi012.stdout:generic/020 3s 2023-07-08T12:49:05.628 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/020 2023-07-08T12:49:05.629 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:49:05.632 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:49:05.952 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:49:05.953 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2023-07-08T12:49:06.394 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:49:06.405 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:49:06.405 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:49:06.407 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:49:06.407 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:49:07.087 INFO:teuthology.orchestra.run.smithi012.stdout:generic/021 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T12:49:07.094 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/021 2023-07-08T12:49:07.094 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/021 2023-07-08T12:49:07.094 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:49:07.097 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:49:07.394 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:49:07.395 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2023-07-08T12:49:07.828 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:49:07.835 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:49:07.836 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:49:07.839 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:49:07.839 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:49:08.508 INFO:teuthology.orchestra.run.smithi012.stdout:generic/022 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T12:49:08.512 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/022 2023-07-08T12:49:08.513 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/022 2023-07-08T12:49:08.513 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:49:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:49:08.850 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:49:08.851 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2023-07-08T12:49:09.229 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:49:09.231 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:49:09.277 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:49:09.286 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:49:09.286 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:49:09.289 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:49:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:49:39.330 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:49:39.332 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:50:09.128 INFO:teuthology.orchestra.run.smithi012.stdout:generic/023 60s 2023-07-08T12:50:09.172 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/023 2023-07-08T12:50:09.173 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:09.408 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:50:09.410 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:50:09.591 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:09.591 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2023-07-08T12:50:09.965 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:09.970 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:09.971 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:09.972 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:09.972 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:10.487 INFO:teuthology.orchestra.run.smithi012.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2023-07-08T12:50:10.492 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/024 2023-07-08T12:50:10.492 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/024 2023-07-08T12:50:10.492 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:10.500 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:10.854 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:10.855 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2023-07-08T12:50:11.296 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:11.307 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:11.308 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:11.308 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:11.308 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:11.847 INFO:teuthology.orchestra.run.smithi012.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2023-07-08T12:50:11.852 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/025 2023-07-08T12:50:11.852 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/025 2023-07-08T12:50:11.852 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:11.862 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:12.214 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:12.215 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2023-07-08T12:50:12.648 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:12.660 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:12.660 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:12.662 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:12.662 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:13.247 INFO:teuthology.orchestra.run.smithi012.stdout:generic/026 [not run] ceph does not define maximum ACL count 2023-07-08T12:50:13.253 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/026 2023-07-08T12:50:13.253 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/026 2023-07-08T12:50:13.253 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:13.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:13.292 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2023-07-08T12:50:13.648 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:13.654 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:13.654 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:13.656 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:13.656 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:14.147 INFO:teuthology.orchestra.run.smithi012.stdout:generic/027 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T12:50:14.153 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/027 2023-07-08T12:50:14.153 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/027 2023-07-08T12:50:14.153 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:14.158 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:14.181 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:14.181 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2023-07-08T12:50:14.575 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:14.580 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:14.581 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:14.582 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:20.116 INFO:teuthology.orchestra.run.smithi012.stdout:generic/028 6s 2023-07-08T12:50:20.122 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/028 2023-07-08T12:50:20.122 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:20.125 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:20.156 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:20.156 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2023-07-08T12:50:20.554 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:20.566 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:20.566 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:20.567 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:20.567 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:24.416 INFO:teuthology.orchestra.run.smithi012.stdout:generic/029 4s 2023-07-08T12:50:24.432 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/029 2023-07-08T12:50:24.432 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:24.447 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:24.448 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2023-07-08T12:50:24.761 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:24.765 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:24.765 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:24.766 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:24.766 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:29.517 INFO:teuthology.orchestra.run.smithi012.stdout:generic/030 4s 2023-07-08T12:50:29.523 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/030 2023-07-08T12:50:29.523 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:29.526 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:29.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:29.543 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2023-07-08T12:50:29.930 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:29.935 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:29.935 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:29.936 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:29.936 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:30.538 INFO:teuthology.orchestra.run.smithi012.stdout:generic/031 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2023-07-08T12:50:30.554 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/031 2023-07-08T12:50:30.554 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/031 2023-07-08T12:50:30.554 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:30.557 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:30.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:30.597 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2023-07-08T12:50:30.965 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:30.975 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:30.976 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:30.977 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:30.977 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:31.451 INFO:teuthology.orchestra.run.smithi012.stdout:generic/032 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T12:50:31.457 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/032 2023-07-08T12:50:31.457 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/032 2023-07-08T12:50:31.457 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:31.497 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:31.497 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2023-07-08T12:50:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:31.834 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:32.456 INFO:teuthology.orchestra.run.smithi012.stdout:generic/033 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-07-08T12:50:32.464 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/033 2023-07-08T12:50:32.464 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/033 2023-07-08T12:50:32.464 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:32.512 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:32.512 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2023-07-08T12:50:32.890 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:32.898 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:32.899 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:32.900 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:32.900 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:33.420 INFO:teuthology.orchestra.run.smithi012.stdout:generic/034 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:50:33.425 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/034 2023-07-08T12:50:33.426 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/034 2023-07-08T12:50:33.426 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:33.428 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:33.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:33.453 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2023-07-08T12:50:33.853 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:33.858 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:33.858 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:33.859 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:33.860 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:34.473 INFO:teuthology.orchestra.run.smithi012.stdout:generic/035 0s 2023-07-08T12:50:34.477 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/035 2023-07-08T12:50:34.477 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:34.480 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:34.780 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:34.781 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2023-07-08T12:50:35.275 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:35.280 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:35.281 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:35.284 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:35.284 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:39.936 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:50:39.938 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:50:45.945 INFO:teuthology.orchestra.run.smithi012.stdout:generic/036 10s 2023-07-08T12:50:45.949 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/036 2023-07-08T12:50:45.950 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:45.951 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:46.199 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:46.199 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2023-07-08T12:50:46.634 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:46.638 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:46.639 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:46.640 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:46.640 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:49.331 INFO:teuthology.orchestra.run.smithi012.stdout:generic/037 3s 2023-07-08T12:50:49.335 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/037 2023-07-08T12:50:49.335 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:49.337 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:49.346 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:49.346 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2023-07-08T12:50:49.570 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:49.575 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:49.575 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:49.576 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:49.576 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:49.924 INFO:teuthology.orchestra.run.smithi012.stdout:generic/038 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T12:50:49.928 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/038 2023-07-08T12:50:49.928 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/038 2023-07-08T12:50:49.928 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:49.953 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:49.954 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2023-07-08T12:50:50.168 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:50.172 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:50.173 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:50.174 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:50.174 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:50.552 INFO:teuthology.orchestra.run.smithi012.stdout:generic/039 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:50:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/039 2023-07-08T12:50:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/039 2023-07-08T12:50:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:50.558 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:50.584 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:50.585 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2023-07-08T12:50:50.805 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:50.809 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:50.810 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:50.811 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:50.811 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:51.159 INFO:teuthology.orchestra.run.smithi012.stdout:generic/040 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:50:51.163 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/040 2023-07-08T12:50:51.163 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/040 2023-07-08T12:50:51.164 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:51.165 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:51.185 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:51.186 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2023-07-08T12:50:51.396 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:51.400 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:51.401 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:51.402 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:51.739 INFO:teuthology.orchestra.run.smithi012.stdout:generic/041 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:50:51.743 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/041 2023-07-08T12:50:51.743 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/041 2023-07-08T12:50:51.743 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:51.767 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:51.767 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2023-07-08T12:50:51.984 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:51.988 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:51.989 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:51.990 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:51.990 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:52.488 INFO:teuthology.orchestra.run.smithi012.stdout:generic/042 [not run] ceph does not support shutdown 2023-07-08T12:50:52.507 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/042 2023-07-08T12:50:52.507 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/042 2023-07-08T12:50:52.508 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:52.509 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:52.532 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:52.532 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2023-07-08T12:50:52.754 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:52.759 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:52.759 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:52.760 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:52.760 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:53.258 INFO:teuthology.orchestra.run.smithi012.stdout:generic/043 [not run] ceph does not support shutdown 2023-07-08T12:50:53.278 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/043 2023-07-08T12:50:53.278 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/043 2023-07-08T12:50:53.279 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:53.280 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:53.303 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:53.303 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2023-07-08T12:50:53.515 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:53.519 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:53.520 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:53.521 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:53.521 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:54.024 INFO:teuthology.orchestra.run.smithi012.stdout:generic/044 [not run] ceph does not support shutdown 2023-07-08T12:50:54.042 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/044 2023-07-08T12:50:54.042 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/044 2023-07-08T12:50:54.042 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:54.044 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:54.065 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:54.065 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2023-07-08T12:50:54.280 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:54.284 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:54.285 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:54.286 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:54.782 INFO:teuthology.orchestra.run.smithi012.stdout:generic/045 [not run] ceph does not support shutdown 2023-07-08T12:50:54.798 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/045 2023-07-08T12:50:54.798 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/045 2023-07-08T12:50:54.798 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:54.800 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:54.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:54.822 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2023-07-08T12:50:55.040 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:55.044 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:55.044 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:55.045 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:55.045 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:55.556 INFO:teuthology.orchestra.run.smithi012.stdout:generic/046 [not run] ceph does not support shutdown 2023-07-08T12:50:55.571 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/046 2023-07-08T12:50:55.572 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/046 2023-07-08T12:50:55.572 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:55.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:55.595 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2023-07-08T12:50:55.816 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:55.820 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:55.821 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:55.822 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:55.822 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:56.305 INFO:teuthology.orchestra.run.smithi012.stdout:generic/047 [not run] ceph does not support shutdown 2023-07-08T12:50:56.321 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/047 2023-07-08T12:50:56.321 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/047 2023-07-08T12:50:56.321 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:56.323 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:56.348 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:56.348 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2023-07-08T12:50:56.562 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:56.566 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:56.566 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:56.567 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:57.071 INFO:teuthology.orchestra.run.smithi012.stdout:generic/048 [not run] ceph does not support shutdown 2023-07-08T12:50:57.086 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/048 2023-07-08T12:50:57.087 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/048 2023-07-08T12:50:57.087 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:57.088 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:57.107 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:57.107 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2023-07-08T12:50:57.330 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:57.335 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:57.335 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:57.336 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:57.336 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:57.848 INFO:teuthology.orchestra.run.smithi012.stdout:generic/049 [not run] ceph does not support shutdown 2023-07-08T12:50:57.859 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/049 2023-07-08T12:50:57.859 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/049 2023-07-08T12:50:57.859 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:57.861 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:57.890 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:57.890 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2023-07-08T12:50:58.107 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:58.111 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:58.112 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:58.113 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:58.113 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:58.595 INFO:teuthology.orchestra.run.smithi012.stdout:generic/050 [not run] ceph does not support shutdown 2023-07-08T12:50:58.610 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/050 2023-07-08T12:50:58.610 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/050 2023-07-08T12:50:58.610 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:58.612 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:58.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:58.634 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2023-07-08T12:50:58.853 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:58.857 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:58.857 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:58.858 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:58.858 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:59.377 INFO:teuthology.orchestra.run.smithi012.stdout:generic/051 [not run] ceph does not support shutdown 2023-07-08T12:50:59.381 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/051 2023-07-08T12:50:59.381 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/051 2023-07-08T12:50:59.381 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:50:59.383 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:50:59.401 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:50:59.402 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2023-07-08T12:50:59.614 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:50:59.618 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:50:59.619 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:50:59.619 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:50:59.620 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:00.123 INFO:teuthology.orchestra.run.smithi012.stdout:generic/052 [not run] ceph does not support shutdown 2023-07-08T12:51:00.139 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/052 2023-07-08T12:51:00.139 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/052 2023-07-08T12:51:00.139 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:00.141 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:00.166 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:00.166 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2023-07-08T12:51:00.386 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:00.390 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:00.391 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:00.392 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:01.511 INFO:teuthology.orchestra.run.smithi012.stdout:generic/053 1s 2023-07-08T12:51:01.532 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/053 2023-07-08T12:51:01.532 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:01.555 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:01.569 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:01.569 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2023-07-08T12:51:01.885 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:01.889 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:01.889 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:01.890 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:01.891 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:02.389 INFO:teuthology.orchestra.run.smithi012.stdout:generic/054 [not run] ceph does not support shutdown 2023-07-08T12:51:02.408 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/054 2023-07-08T12:51:02.408 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/054 2023-07-08T12:51:02.408 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:02.410 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:02.431 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:02.431 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2023-07-08T12:51:02.658 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:02.662 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:02.663 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:02.664 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:02.664 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:03.175 INFO:teuthology.orchestra.run.smithi012.stdout:generic/055 [not run] ceph does not support shutdown 2023-07-08T12:51:03.191 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/055 2023-07-08T12:51:03.191 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/055 2023-07-08T12:51:03.191 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:03.193 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:03.225 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:03.225 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2023-07-08T12:51:03.451 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:03.456 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:03.456 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:03.457 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:03.457 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:03.822 INFO:teuthology.orchestra.run.smithi012.stdout:generic/056 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:51:03.826 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/056 2023-07-08T12:51:03.826 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/056 2023-07-08T12:51:03.826 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:03.828 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:03.854 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:03.855 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2023-07-08T12:51:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:04.082 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:04.082 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:04.083 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:04.084 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:04.431 INFO:teuthology.orchestra.run.smithi012.stdout:generic/057 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:51:04.437 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/057 2023-07-08T12:51:04.438 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/057 2023-07-08T12:51:04.438 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:04.438 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:04.467 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:04.467 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2023-07-08T12:51:04.687 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:04.691 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:04.691 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:04.693 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:05.132 INFO:teuthology.orchestra.run.smithi012.stdout:generic/058 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T12:51:05.135 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/058 2023-07-08T12:51:05.135 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/058 2023-07-08T12:51:05.135 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:05.360 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:05.361 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2023-07-08T12:51:05.745 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:05.749 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:05.750 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:05.751 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:05.751 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:06.119 INFO:teuthology.orchestra.run.smithi012.stdout:generic/059 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:51:06.122 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/059 2023-07-08T12:51:06.123 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/059 2023-07-08T12:51:06.123 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:06.125 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:06.144 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:06.144 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2023-07-08T12:51:06.365 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:06.370 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:06.370 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:06.371 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:06.371 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:06.808 INFO:teuthology.orchestra.run.smithi012.stdout:generic/060 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T12:51:06.812 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/060 2023-07-08T12:51:06.812 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/060 2023-07-08T12:51:06.812 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:06.814 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:07.157 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:07.157 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2023-07-08T12:51:07.406 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:07.410 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:07.410 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:07.411 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:07.412 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:07.836 INFO:teuthology.orchestra.run.smithi012.stdout:generic/061 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T12:51:07.840 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/061 2023-07-08T12:51:07.840 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/061 2023-07-08T12:51:07.840 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:07.842 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:08.142 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:08.143 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2023-07-08T12:51:08.418 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:08.422 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:08.423 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:08.424 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:10.030 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:51:10.140 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:51:10.313 INFO:teuthology.orchestra.run.smithi012.stdout:generic/062 2s 2023-07-08T12:51:10.320 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/062 2023-07-08T12:51:10.321 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:10.323 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:10.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:10.332 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2023-07-08T12:51:10.546 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:10.550 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:10.550 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:10.551 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:10.551 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:11.030 INFO:teuthology.orchestra.run.smithi012.stdout:generic/063 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T12:51:11.034 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/063 2023-07-08T12:51:11.034 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/063 2023-07-08T12:51:11.034 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:11.036 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:11.398 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:11.398 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2023-07-08T12:51:11.653 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:11.657 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:11.657 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:11.658 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:11.659 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:12.007 INFO:teuthology.orchestra.run.smithi012.stdout:generic/064 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-07-08T12:51:12.011 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/064 2023-07-08T12:51:12.011 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/064 2023-07-08T12:51:12.011 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:12.013 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:12.037 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:12.038 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2023-07-08T12:51:12.258 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:12.262 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:12.262 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:12.263 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:12.264 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:12.653 INFO:teuthology.orchestra.run.smithi012.stdout:generic/065 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:51:12.657 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/065 2023-07-08T12:51:12.657 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/065 2023-07-08T12:51:12.657 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:12.659 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:12.683 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:12.683 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2023-07-08T12:51:12.900 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:12.904 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:12.904 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:12.905 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:12.905 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:13.251 INFO:teuthology.orchestra.run.smithi012.stdout:generic/066 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:51:13.256 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/066 2023-07-08T12:51:13.256 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/066 2023-07-08T12:51:13.256 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:13.285 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:13.285 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2023-07-08T12:51:13.503 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:13.507 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:13.508 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:13.509 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:13.509 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:13.887 INFO:teuthology.orchestra.run.smithi012.stdout:generic/067 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:51:13.891 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/067 2023-07-08T12:51:13.891 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/067 2023-07-08T12:51:13.891 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:13.893 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:13.924 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:13.924 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2023-07-08T12:51:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:14.148 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:14.149 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:14.150 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:14.150 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:14.511 INFO:teuthology.orchestra.run.smithi012.stdout:generic/068 [not run] ceph does not support freezing 2023-07-08T12:51:14.517 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/068 2023-07-08T12:51:14.517 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/068 2023-07-08T12:51:14.517 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:14.539 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:14.540 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2023-07-08T12:51:14.765 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:14.769 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:14.770 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:14.771 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:21.631 INFO:teuthology.orchestra.run.smithi012.stdout:generic/069 7s 2023-07-08T12:51:21.636 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/069 2023-07-08T12:51:21.636 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:21.638 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:21.651 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:21.651 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2023-07-08T12:51:21.955 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:21.959 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:21.960 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:21.961 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:21.961 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:40.363 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:51:40.417 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:51:49.176 INFO:teuthology.orchestra.run.smithi012.stdout:generic/070 27s 2023-07-08T12:51:49.200 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/070 2023-07-08T12:51:49.200 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:49.203 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:49.513 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:49.513 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2023-07-08T12:51:50.110 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:50.117 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:50.118 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:50.119 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:50.119 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:50.635 INFO:teuthology.orchestra.run.smithi012.stdout:generic/071 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-07-08T12:51:50.640 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/071 2023-07-08T12:51:50.640 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/071 2023-07-08T12:51:50.641 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:50.642 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:50.678 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:50.678 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2023-07-08T12:51:51.053 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:51.058 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:51.058 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:51.060 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:51.060 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:51.672 INFO:teuthology.orchestra.run.smithi012.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2023-07-08T12:51:51.676 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/072 2023-07-08T12:51:51.677 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/072 2023-07-08T12:51:51.677 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:51.680 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:51.710 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:51.710 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2023-07-08T12:51:52.104 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:52.111 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:52.112 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:52.113 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:52.113 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:52.620 INFO:teuthology.orchestra.run.smithi012.stdout:generic/073 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:51:52.627 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/073 2023-07-08T12:51:52.627 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/073 2023-07-08T12:51:52.627 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:51:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:51:52.654 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:51:52.655 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2023-07-08T12:51:53.039 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:51:53.047 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:51:53.048 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:51:53.049 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:51:53.049 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:52:10.773 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:52:10.778 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:52:10.893 INFO:tasks.ceph.mds.c.smithi012.stderr:2023-07-08T12:52:10.891+0000 7f187cc06700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 239039) UID: 0 2023-07-08T12:52:10.894 INFO:tasks.ceph.mds.a.smithi012.stderr:2023-07-08T12:52:10.891+0000 7f47874c4700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 239039) UID: 0 2023-07-08T12:52:14.697 INFO:teuthology.orchestra.run.smithi012.stdout:generic/074 21s 2023-07-08T12:52:14.773 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/074 2023-07-08T12:52:14.773 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:52:14.781 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:52:15.766 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:52:15.767 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2023-07-08T12:52:16.812 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:52:16.821 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:52:16.822 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:52:16.824 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:52:16.824 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:52:54.370 INFO:teuthology.orchestra.run.smithi012.stdout:generic/075 37s 2023-07-08T12:52:54.379 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/075 2023-07-08T12:52:54.380 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:52:54.382 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:52:54.682 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:52:54.682 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2023-07-08T12:52:55.393 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:52:55.402 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:52:55.404 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:52:55.406 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:52:55.406 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:52:55.997 INFO:teuthology.orchestra.run.smithi012.stdout:generic/076 [not run] require 172.21.15.12:6789:/scratch to be local device 2023-07-08T12:52:56.006 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/076 2023-07-08T12:52:56.007 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/076 2023-07-08T12:52:56.007 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:52:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:52:56.058 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:52:56.058 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2023-07-08T12:52:56.549 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:52:56.554 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:52:56.555 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:52:56.560 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:52:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:05.399 INFO:teuthology.orchestra.run.smithi012.stdout:generic/077 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T12:53:05.411 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/077 2023-07-08T12:53:05.411 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/077 2023-07-08T12:53:05.411 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:53:05.431 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:05.885 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:53:05.886 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2023-07-08T12:53:06.442 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:53:06.448 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:53:06.448 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:53:06.449 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:53:06.450 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:07.117 INFO:teuthology.orchestra.run.smithi012.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2023-07-08T12:53:07.129 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/078 2023-07-08T12:53:07.129 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/078 2023-07-08T12:53:07.130 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:53:07.132 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:07.382 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:53:07.382 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2023-07-08T12:53:08.078 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:53:08.100 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:53:08.100 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:53:08.102 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:53:08.102 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:08.756 INFO:teuthology.orchestra.run.smithi012.stdout:generic/079 [not run] file system doesn't support chattr +ia 2023-07-08T12:53:08.761 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/079 2023-07-08T12:53:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/079 2023-07-08T12:53:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:53:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:08.815 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:53:08.816 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2023-07-08T12:53:09.315 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:53:09.321 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:53:09.324 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:53:09.325 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:53:09.325 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:11.974 INFO:teuthology.orchestra.run.smithi012.stdout:generic/080 2s 2023-07-08T12:53:12.001 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/080 2023-07-08T12:53:12.001 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:53:12.004 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:12.061 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:53:12.061 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2023-07-08T12:53:12.578 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:53:12.586 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:53:12.591 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:53:12.592 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:53:12.593 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:13.237 INFO:teuthology.orchestra.run.smithi012.stdout:generic/081 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:53:13.242 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/081 2023-07-08T12:53:13.242 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/081 2023-07-08T12:53:13.242 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:53:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:13.286 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:53:13.287 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2023-07-08T12:53:13.708 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:53:13.726 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:53:13.726 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:53:13.729 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:53:13.729 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:14.314 INFO:teuthology.orchestra.run.smithi012.stdout:generic/082 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T12:53:14.321 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/082 2023-07-08T12:53:14.321 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/082 2023-07-08T12:53:14.321 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:53:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:14.354 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:53:14.354 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2023-07-08T12:53:14.840 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:53:14.846 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:53:14.846 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:53:14.850 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:53:14.851 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:15.452 INFO:teuthology.orchestra.run.smithi012.stdout:generic/083 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T12:53:15.462 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/083 2023-07-08T12:53:15.462 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/083 2023-07-08T12:53:15.462 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:53:15.465 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:15.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:53:15.505 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2023-07-08T12:53:15.952 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:53:15.973 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:53:15.973 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:53:15.975 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:53:15.975 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:22.436 INFO:teuthology.orchestra.run.smithi012.stdout:generic/084 6s 2023-07-08T12:53:22.466 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/084 2023-07-08T12:53:22.467 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:53:22.469 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:22.482 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:53:22.482 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2023-07-08T12:53:23.009 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:53:23.017 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:53:23.018 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:23.665 INFO:teuthology.orchestra.run.smithi012.stdout:generic/085 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T12:53:23.669 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/085 2023-07-08T12:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/085 2023-07-08T12:53:23.670 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:53:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:23.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:53:23.734 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2023-07-08T12:53:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:53:24.250 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:53:24.251 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:53:24.253 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:53:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:24.863 INFO:teuthology.orchestra.run.smithi012.stdout:generic/086 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T12:53:24.868 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/086 2023-07-08T12:53:24.868 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/086 2023-07-08T12:53:24.868 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:53:24.889 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:24.933 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:53:24.934 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2023-07-08T12:53:25.461 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:53:25.470 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:53:25.471 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:53:25.472 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:53:25.472 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:26.551 INFO:teuthology.orchestra.run.smithi012.stdout:generic/087 1s 2023-07-08T12:53:26.582 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/087 2023-07-08T12:53:26.582 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:53:26.586 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:27.048 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:53:27.049 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2023-07-08T12:53:27.501 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:53:27.506 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:53:27.507 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:53:27.508 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:53:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:28.346 INFO:teuthology.orchestra.run.smithi012.stdout:generic/088 1s 2023-07-08T12:53:28.424 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/088 2023-07-08T12:53:28.425 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T12:53:28.427 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:53:28.788 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T12:53:28.789 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2023-07-08T12:53:29.366 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T12:53:29.384 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T12:53:29.385 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T12:53:29.386 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T12:53:29.386 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T12:55:43.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:55:43.978 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:56:14.373 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:56:14.375 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:56:44.440 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:56:44.442 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:57:14.919 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:57:14.921 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:57:44.983 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:57:44.985 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T12:57:45.023 INFO:tasks.ceph.mds.a.smithi012.stderr:2023-07-08T12:57:45.021+0000 7f47874c4700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 248583) UID: 0 2023-07-08T12:57:45.024 INFO:tasks.ceph.mds.c.smithi012.stderr:2023-07-08T12:57:45.021+0000 7f187cc06700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 248583) UID: 0 2023-07-08T13:00:46.009 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:00:46.016 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:01:16.486 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:01:16.488 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:01:46.558 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:01:46.581 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:01:46.864 INFO:tasks.ceph.mds.a.smithi012.stderr:2023-07-08T13:01:46.862+0000 7f47874c4700 -1 received signal: Hangup from (PID: 250726) UID: 0 2023-07-08T13:01:46.868 INFO:tasks.ceph.mds.c.smithi012.stderr:2023-07-08T13:01:46.862+0000 7f187cc06700 -1 Fail to open '/proc/250726/cmdline' error = (2) No such file or directory 2023-07-08T13:01:46.868 INFO:tasks.ceph.mds.c.smithi012.stderr:2023-07-08T13:01:46.862+0000 7f187cc06700 -1 received signal: Hangup from (PID: 250726) UID: 0 2023-07-08T13:01:56.939 INFO:teuthology.orchestra.run.smithi012.stdout:generic/089 507s 2023-07-08T13:01:56.944 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/089 2023-07-08T13:01:56.944 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:01:56.946 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:01:58.429 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:01:58.430 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2023-07-08T13:01:59.352 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:01:59.356 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:01:59.356 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:01:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:01:59.358 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:01:59.742 INFO:teuthology.orchestra.run.smithi012.stdout:generic/090 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:01:59.745 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/090 2023-07-08T13:01:59.746 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/090 2023-07-08T13:01:59.746 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:01:59.748 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:01:59.770 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:01:59.770 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2023-07-08T13:01:59.997 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:02:00.001 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:02:00.001 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:02:00.002 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:02:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:03:44.448 INFO:teuthology.orchestra.run.smithi012.stdout:generic/091 104s 2023-07-08T13:03:44.452 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/091 2023-07-08T13:03:44.453 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:03:44.455 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:03:44.879 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:03:44.880 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2023-07-08T13:03:45.110 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:03:45.114 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:03:45.115 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:03:45.116 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:03:45.116 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:03:45.474 INFO:teuthology.orchestra.run.smithi012.stdout:generic/092 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T13:03:45.478 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/092 2023-07-08T13:03:45.478 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/092 2023-07-08T13:03:45.478 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:03:45.480 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:03:45.505 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:03:45.505 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2023-07-08T13:03:45.723 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:03:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:03:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:03:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:03:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:03:46.363 INFO:teuthology.orchestra.run.smithi012.stdout:generic/093 1s 2023-07-08T13:03:46.367 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/093 2023-07-08T13:03:46.367 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:03:46.369 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:03:46.787 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:03:46.787 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2023-07-08T13:03:47.034 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:03:47.039 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:03:47.039 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:03:47.040 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:03:47.040 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:03:47.417 INFO:teuthology.orchestra.run.smithi012.stdout:generic/094 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-07-08T13:03:47.421 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/094 2023-07-08T13:03:47.421 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/094 2023-07-08T13:03:47.421 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:03:47.423 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:03:47.863 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:03:47.863 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2023-07-08T13:03:48.096 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:03:48.100 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:03:48.100 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:03:48.101 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:03:48.101 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:01.051 INFO:teuthology.orchestra.run.smithi012.stdout:generic/095 13s 2023-07-08T13:04:01.058 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/095 2023-07-08T13:04:01.058 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:01.061 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:01.074 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:01.074 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2023-07-08T13:04:01.297 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:01.301 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:01.302 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:01.303 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:01.303 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:01.723 INFO:teuthology.orchestra.run.smithi012.stdout:generic/096 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-07-08T13:04:01.727 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/096 2023-07-08T13:04:01.727 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/096 2023-07-08T13:04:01.727 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:01.729 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:01.763 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:01.763 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2023-07-08T13:04:01.980 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:01.984 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:01.984 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:01.985 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:01.986 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:03.186 INFO:teuthology.orchestra.run.smithi012.stdout:generic/097 0s 2023-07-08T13:04:03.251 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/097 2023-07-08T13:04:03.252 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:03.254 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:03.429 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:03.430 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2023-07-08T13:04:03.652 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:03.656 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:03.657 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:03.658 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:03.658 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:06.155 INFO:teuthology.orchestra.run.smithi012.stdout:generic/098 3s 2023-07-08T13:04:06.160 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/098 2023-07-08T13:04:06.160 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:06.162 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:06.170 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:06.171 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2023-07-08T13:04:06.390 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:06.394 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:06.395 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:06.396 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:06.396 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:07.360 INFO:teuthology.orchestra.run.smithi012.stdout:generic/099 1s 2023-07-08T13:04:07.365 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/099 2023-07-08T13:04:07.365 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:07.367 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:07.756 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:07.757 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2023-07-08T13:04:08.018 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:08.022 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:08.023 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:08.024 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:08.024 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:20.424 INFO:teuthology.orchestra.run.smithi012.stdout:generic/100 12s 2023-07-08T13:04:20.443 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/100 2023-07-08T13:04:20.443 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:20.481 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:23.995 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:23.995 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2023-07-08T13:04:24.464 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:24.468 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:24.469 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:24.469 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:24.470 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:24.811 INFO:teuthology.orchestra.run.smithi012.stdout:generic/101 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:04:24.815 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/101 2023-07-08T13:04:24.815 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/101 2023-07-08T13:04:24.815 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:24.817 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:24.835 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:24.836 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2023-07-08T13:04:25.057 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:25.062 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:25.062 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:25.063 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:25.406 INFO:teuthology.orchestra.run.smithi012.stdout:generic/102 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T13:04:25.410 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/102 2023-07-08T13:04:25.411 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/102 2023-07-08T13:04:25.411 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:25.412 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:25.440 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:25.440 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2023-07-08T13:04:25.661 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:25.665 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:25.666 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:25.666 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:25.667 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:26.016 INFO:teuthology.orchestra.run.smithi012.stdout:generic/103 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T13:04:26.020 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/103 2023-07-08T13:04:26.020 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/103 2023-07-08T13:04:26.020 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:26.022 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:26.399 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:26.399 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2023-07-08T13:04:26.647 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:26.651 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:26.651 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:26.652 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:26.653 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:27.010 INFO:teuthology.orchestra.run.smithi012.stdout:generic/104 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:04:27.014 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/104 2023-07-08T13:04:27.015 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/104 2023-07-08T13:04:27.015 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:27.050 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:27.051 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2023-07-08T13:04:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:27.279 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:27.280 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:27.281 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:27.281 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:28.279 INFO:teuthology.orchestra.run.smithi012.stdout:generic/105 1s 2023-07-08T13:04:28.284 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/105 2023-07-08T13:04:28.285 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:28.287 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:28.295 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:28.296 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2023-07-08T13:04:28.521 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:28.526 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:28.526 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:28.527 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:28.903 INFO:teuthology.orchestra.run.smithi012.stdout:generic/106 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:04:28.907 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/106 2023-07-08T13:04:28.907 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/106 2023-07-08T13:04:28.907 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:28.909 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:28.940 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:28.940 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2023-07-08T13:04:29.168 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:29.173 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:29.173 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:29.174 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:29.175 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:29.536 INFO:teuthology.orchestra.run.smithi012.stdout:generic/107 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:04:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/107 2023-07-08T13:04:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/107 2023-07-08T13:04:29.540 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:29.542 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:29.571 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:29.572 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2023-07-08T13:04:29.799 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:29.803 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:29.804 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:29.805 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:29.805 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:30.271 INFO:teuthology.orchestra.run.smithi012.stdout:generic/108 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:04:30.275 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/108 2023-07-08T13:04:30.275 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/108 2023-07-08T13:04:30.275 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:30.305 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:30.306 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2023-07-08T13:04:30.528 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:30.532 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:30.533 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:30.534 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:34.772 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:04:34.774 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:04:40.857 INFO:teuthology.orchestra.run.smithi012.stdout:generic/109 10s 2023-07-08T13:04:40.897 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/109 2023-07-08T13:04:40.897 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:04:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:04:40.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:04:40.909 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2023-07-08T13:04:41.891 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:04:41.896 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:04:41.896 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:04:41.897 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:04:41.897 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:03.003 INFO:teuthology.orchestra.run.smithi012.stdout:generic/110 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:05:03.007 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/110 2023-07-08T13:05:03.007 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/110 2023-07-08T13:05:03.007 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:05:03.009 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:03.366 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:05:03.367 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2023-07-08T13:05:04.055 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:05:04.060 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:05:04.060 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:05:04.061 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:05:04.061 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:04.414 INFO:teuthology.orchestra.run.smithi012.stdout:generic/111 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:05:04.418 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/111 2023-07-08T13:05:04.419 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/111 2023-07-08T13:05:04.419 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:05:04.421 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:04.589 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:05:04.590 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2023-07-08T13:05:04.877 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:05:04.879 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:05:05.057 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:05:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:05:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:05:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:05:05.063 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:21.129 INFO:teuthology.orchestra.run.smithi012.stdout:generic/112 16s 2023-07-08T13:05:21.189 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/112 2023-07-08T13:05:21.190 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:05:21.192 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:21.528 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:05:21.528 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2023-07-08T13:05:21.819 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:05:21.823 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:05:21.824 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:05:21.825 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:05:21.825 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:34.947 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:05:34.980 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:05:35.872 INFO:teuthology.orchestra.run.smithi012.stdout:generic/113 13s 2023-07-08T13:05:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/113 2023-07-08T13:05:35.883 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:05:35.885 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:36.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:05:36.151 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2023-07-08T13:05:36.657 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:05:36.661 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:05:36.661 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:05:36.662 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:05:36.663 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:37.062 INFO:teuthology.orchestra.run.smithi012.stdout:generic/114 [not run] device block size: 4096 greater than 512 2023-07-08T13:05:37.066 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/114 2023-07-08T13:05:37.066 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/114 2023-07-08T13:05:37.066 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:05:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:37.104 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:05:37.104 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2023-07-08T13:05:37.325 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:05:37.329 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:05:37.329 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:05:37.330 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:05:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:37.696 INFO:teuthology.orchestra.run.smithi012.stdout:generic/115 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:05:37.700 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/115 2023-07-08T13:05:37.700 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/115 2023-07-08T13:05:37.700 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:05:37.702 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:37.787 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:05:37.787 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2023-07-08T13:05:38.323 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:05:38.327 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:05:38.328 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:05:38.329 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:05:38.329 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:38.697 INFO:teuthology.orchestra.run.smithi012.stdout:generic/116 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:05:38.701 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/116 2023-07-08T13:05:38.701 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/116 2023-07-08T13:05:38.701 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:05:38.703 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:39.124 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:05:39.124 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2023-07-08T13:05:39.353 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:05:39.358 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:05:39.358 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:05:39.359 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:05:39.359 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:56.355 INFO:teuthology.orchestra.run.smithi012.stdout:generic/117 17s 2023-07-08T13:05:56.364 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/117 2023-07-08T13:05:56.365 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:05:56.367 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:56.376 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:05:56.376 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2023-07-08T13:05:56.608 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:05:56.612 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:05:56.613 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:05:56.614 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:05:56.614 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:57.419 INFO:teuthology.orchestra.run.smithi012.stdout:generic/118 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:05:57.423 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/118 2023-07-08T13:05:57.423 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/118 2023-07-08T13:05:57.423 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:05:57.425 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:57.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:05:57.813 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2023-07-08T13:05:58.072 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:05:58.077 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:05:58.077 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:05:58.078 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:05:58.078 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:58.435 INFO:teuthology.orchestra.run.smithi012.stdout:generic/119 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:05:58.439 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/119 2023-07-08T13:05:58.439 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/119 2023-07-08T13:05:58.439 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:05:58.441 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:58.839 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:05:58.839 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2023-07-08T13:05:59.084 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:05:59.089 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:05:59.089 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:05:59.090 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:05:59.090 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:59.425 INFO:teuthology.orchestra.run.smithi012.stdout:generic/120 [not run] atime not maintained by ceph 2023-07-08T13:05:59.429 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/120 2023-07-08T13:05:59.429 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/120 2023-07-08T13:05:59.430 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:05:59.431 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:05:59.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:05:59.454 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2023-07-08T13:05:59.671 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:05:59.676 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:05:59.676 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:05:59.677 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:05:59.677 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:06:00.047 INFO:teuthology.orchestra.run.smithi012.stdout:generic/121 [not run] Dedupe not supported by test filesystem type: ceph 2023-07-08T13:06:00.051 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/121 2023-07-08T13:06:00.051 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/121 2023-07-08T13:06:00.051 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:06:00.053 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:06:00.463 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:06:00.463 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2023-07-08T13:06:00.703 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:06:00.707 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:06:00.707 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:06:00.708 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:06:00.709 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:06:01.086 INFO:teuthology.orchestra.run.smithi012.stdout:generic/122 [not run] Dedupe not supported by test filesystem type: ceph 2023-07-08T13:06:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/122 2023-07-08T13:06:01.091 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/122 2023-07-08T13:06:01.091 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:06:01.093 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:06:01.506 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:06:01.506 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2023-07-08T13:06:01.739 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:06:01.743 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:06:01.743 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:06:01.744 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:06:01.744 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:06:02.229 INFO:teuthology.orchestra.run.smithi012.stdout:generic/123 1s 2023-07-08T13:06:02.234 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/123 2023-07-08T13:06:02.234 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:06:02.236 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:06:02.407 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:06:02.408 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2023-07-08T13:06:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:06:02.879 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:06:02.880 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:06:02.881 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:06:02.881 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:06:05.240 INFO:teuthology.orchestra.run.smithi012.stdout:generic/124 2s 2023-07-08T13:06:05.245 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/124 2023-07-08T13:06:05.245 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:06:05.247 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:06:05.302 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:06:05.368 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:06:05.814 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:06:05.814 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2023-07-08T13:06:06.147 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:06:06.152 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:06:06.152 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:06:06.153 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:06:06.153 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:06:35.495 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:06:35.498 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:07:05.580 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:07:05.583 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:07:07.750 INFO:teuthology.orchestra.run.smithi012.stdout:generic/125 61s 2023-07-08T13:07:07.756 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/125 2023-07-08T13:07:07.756 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:07:07.758 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:07:07.982 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:07:07.983 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2023-07-08T13:07:08.415 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:07:08.419 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:07:08.420 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:07:08.421 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:07:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:07:08.927 INFO:teuthology.orchestra.run.smithi012.stdout:generic/126 0s 2023-07-08T13:07:08.932 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/126 2023-07-08T13:07:08.932 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:07:08.934 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:07:09.355 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:07:09.355 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2023-07-08T13:07:09.615 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:07:09.619 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:07:09.619 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:07:09.620 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:07:09.621 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:07:35.680 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:07:35.682 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:08:05.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:08:05.843 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:08:35.903 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:08:35.906 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:09:05.983 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:09:05.985 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:09:36.037 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:09:36.040 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:10:06.101 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:10:06.103 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:10:36.217 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:10:36.219 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:11:06.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:11:06.648 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:11:37.050 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:11:37.052 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:12:07.122 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:12:07.125 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:12:37.181 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:12:37.184 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:13:07.255 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:13:07.257 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:13:37.322 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:13:37.324 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:14:07.469 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:14:07.471 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:14:37.550 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:14:37.552 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:15:07.621 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:15:07.623 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:15:38.035 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:15:38.038 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:16:08.364 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:16:08.366 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:16:28.356 INFO:teuthology.orchestra.run.smithi012.stdout:generic/127 559s 2023-07-08T13:16:28.361 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/127 2023-07-08T13:16:28.361 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:16:28.363 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:16:28.713 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:16:28.714 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2023-07-08T13:16:28.994 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:16:28.998 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:16:28.999 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:16:29.000 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:16:29.000 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:16:30.218 INFO:teuthology.orchestra.run.smithi012.stdout:generic/128 1s 2023-07-08T13:16:30.223 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/128 2023-07-08T13:16:30.223 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:16:30.225 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:16:30.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:16:30.234 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2023-07-08T13:16:30.449 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:16:30.453 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:16:30.454 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:16:30.455 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:16:30.455 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:16:38.634 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:16:38.637 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:16:38.681 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-07-08T13:16:38.680+0000 7f83fa332700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 290809) UID: 0 2023-07-08T13:16:38.681 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-07-08T13:16:38.680+0000 7fe6c1b3c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 290809) UID: 0 2023-07-08T13:16:38.682 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-07-08T13:16:38.680+0000 7fcf305dc700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 290809) UID: 0 2023-07-08T13:16:38.682 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-07-08T13:16:38.680+0000 7f589b052700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 290809) UID: 0 2023-07-08T13:20:35.114 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:20:35.122 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:20:35.266 INFO:tasks.ceph.osd.7.smithi160.stderr:2023-07-08T13:20:35.265+0000 7f0f5014f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 113791) UID: 0 2023-07-08T13:20:35.267 INFO:tasks.ceph.osd.4.smithi160.stderr:2023-07-08T13:20:35.265+0000 7efd6f6e9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 113791) UID: 0 2023-07-08T13:20:35.270 INFO:tasks.ceph.osd.5.smithi160.stderr:2023-07-08T13:20:35.268+0000 7f3d3601f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 113791) UID: 0 2023-07-08T13:20:35.276 INFO:tasks.ceph.osd.6.smithi160.stderr:2023-07-08T13:20:35.269+0000 7fb3c0965700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 113791) UID: 0 2023-07-08T13:25:05.280 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:25:05.316 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:25:05.552 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-07-08T13:25:05.550+0000 7fe6c1b3c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 290938) UID: 0 2023-07-08T13:25:05.557 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-07-08T13:25:05.555+0000 7f589b052700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 290938) UID: 0 2023-07-08T13:25:05.569 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-07-08T13:25:05.555+0000 7f83fa332700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 290938) UID: 0 2023-07-08T13:25:05.571 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-07-08T13:25:05.555+0000 7fcf305dc700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 290938) UID: 0 2023-07-08T13:30:08.954 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:30:08.999 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:30:09.119 INFO:tasks.ceph.osd.6.smithi160.stderr:2023-07-08T13:30:09.118+0000 7fb3c0965700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 113941) UID: 0 2023-07-08T13:30:09.120 INFO:tasks.ceph.osd.7.smithi160.stderr:2023-07-08T13:30:09.118+0000 7f0f5014f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 113941) UID: 0 2023-07-08T13:30:09.120 INFO:tasks.ceph.osd.5.smithi160.stderr:2023-07-08T13:30:09.118+0000 7f3d3601f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 113941) UID: 0 2023-07-08T13:30:09.120 INFO:tasks.ceph.osd.4.smithi160.stderr:2023-07-08T13:30:09.118+0000 7efd6f6e9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 113941) UID: 0 2023-07-08T13:30:09.292 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-07-08T13:30:09.287+0000 7fe6c1b3c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 291046) UID: 0 2023-07-08T13:30:09.292 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-07-08T13:30:09.290+0000 7fcf305dc700 -1 Fail to open '/proc/291046/cmdline' error = (2) No such file or directory 2023-07-08T13:30:09.293 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-07-08T13:30:09.290+0000 7fcf305dc700 -1 received signal: Hangup from (PID: 291046) UID: 0 2023-07-08T13:30:09.293 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-07-08T13:30:09.291+0000 7f83fa332700 -1 Fail to open '/proc/291046/cmdline' error = (2) No such file or directory 2023-07-08T13:30:09.293 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-07-08T13:30:09.291+0000 7f83fa332700 -1 received signal: Hangup from (PID: 291046) UID: 0 2023-07-08T13:30:09.293 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-07-08T13:30:09.291+0000 7f589b052700 -1 Fail to open '/proc/291046/cmdline' error = (2) No such file or directory 2023-07-08T13:30:09.294 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-07-08T13:30:09.291+0000 7f589b052700 -1 received signal: Hangup from (PID: 291046) UID: 0 2023-07-08T13:35:05.843 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:35:05.886 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:35:06.031 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-07-08T13:35:06.030+0000 7f83fa332700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 291110) UID: 0 2023-07-08T13:35:06.032 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-07-08T13:35:06.030+0000 7fe6c1b3c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 291110) UID: 0 2023-07-08T13:35:06.032 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-07-08T13:35:06.030+0000 7fcf305dc700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 291110) UID: 0 2023-07-08T13:35:06.032 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-07-08T13:35:06.030+0000 7f589b052700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 291110) UID: 0 2023-07-08T13:35:06.038 INFO:tasks.ceph.osd.6.smithi160.stderr:2023-07-08T13:35:06.035+0000 7fb3c0965700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 114011) UID: 0 2023-07-08T13:35:06.038 INFO:tasks.ceph.osd.7.smithi160.stderr:2023-07-08T13:35:06.035+0000 7f0f5014f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 114011) UID: 0 2023-07-08T13:35:06.038 INFO:tasks.ceph.osd.4.smithi160.stderr:2023-07-08T13:35:06.035+0000 7efd6f6e9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 114011) UID: 0 2023-07-08T13:35:06.039 INFO:tasks.ceph.osd.5.smithi160.stderr:2023-07-08T13:35:06.035+0000 7f3d3601f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 114011) UID: 0 2023-07-08T13:37:29.978 INFO:teuthology.orchestra.run.smithi012.stdout:generic/129 1259s 2023-07-08T13:37:30.022 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/129 2023-07-08T13:37:30.022 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:37:30.024 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:37:30.033 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:37:30.034 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2023-07-08T13:37:30.418 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:37:30.422 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:37:30.422 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:37:30.423 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:37:30.424 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:37:43.184 INFO:teuthology.orchestra.run.smithi012.stdout:generic/130 13s 2023-07-08T13:37:43.192 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/130 2023-07-08T13:37:43.192 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:37:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:37:43.207 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:37:43.208 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2023-07-08T13:37:43.437 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:37:43.442 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:37:43.442 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:37:43.443 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:37:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:37:45.006 INFO:teuthology.orchestra.run.smithi012.stdout:generic/131 2s 2023-07-08T13:37:45.016 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/131 2023-07-08T13:37:45.016 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:37:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:37:45.448 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:37:45.448 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2023-07-08T13:37:46.009 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:37:46.034 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:37:46.037 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:37:46.038 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:37:46.038 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:38:00.700 INFO:teuthology.orchestra.run.smithi012.stdout:generic/132 14s 2023-07-08T13:38:00.705 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/132 2023-07-08T13:38:00.705 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:38:00.707 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:38:00.715 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:38:00.716 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2023-07-08T13:38:00.952 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:38:00.956 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:38:00.957 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:38:00.958 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:38:00.958 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:41:18.012 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:41:18.015 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:41:48.124 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:41:48.127 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:42:00.108 INFO:teuthology.orchestra.run.smithi012.stdout:generic/133 239s 2023-07-08T13:42:00.112 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/133 2023-07-08T13:42:00.112 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:00.114 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:00.392 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:00.393 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2023-07-08T13:42:01.005 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:01.009 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:01.009 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:01.010 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:01.010 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:01.398 INFO:teuthology.orchestra.run.smithi012.stdout:generic/134 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:01.402 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/134 2023-07-08T13:42:01.402 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/134 2023-07-08T13:42:01.402 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:01.404 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:01.815 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:01.816 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2023-07-08T13:42:02.091 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:02.096 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:02.096 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:02.097 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:02.098 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:03.271 INFO:teuthology.orchestra.run.smithi012.stdout:generic/135 1s 2023-07-08T13:42:03.276 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/135 2023-07-08T13:42:03.276 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:03.287 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:03.287 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2023-07-08T13:42:03.515 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:03.519 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:03.520 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:03.521 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:03.521 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:03.889 INFO:teuthology.orchestra.run.smithi012.stdout:generic/136 [not run] Dedupe not supported by test filesystem type: ceph 2023-07-08T13:42:03.893 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/136 2023-07-08T13:42:03.893 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/136 2023-07-08T13:42:03.893 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:03.895 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:04.128 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:04.128 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2023-07-08T13:42:04.564 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:04.568 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:04.568 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:04.569 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:04.569 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:04.946 INFO:teuthology.orchestra.run.smithi012.stdout:generic/137 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:04.949 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/137 2023-07-08T13:42:04.950 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/137 2023-07-08T13:42:04.950 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:04.952 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:05.346 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:05.347 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2023-07-08T13:42:05.628 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:05.633 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:05.633 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:05.634 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:05.634 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:06.005 INFO:teuthology.orchestra.run.smithi012.stdout:generic/138 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:06.009 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/138 2023-07-08T13:42:06.009 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/138 2023-07-08T13:42:06.010 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:06.011 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:06.338 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:06.338 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2023-07-08T13:42:06.693 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:06.697 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:06.698 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:06.699 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:06.699 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:07.068 INFO:teuthology.orchestra.run.smithi012.stdout:generic/139 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:07.072 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/139 2023-07-08T13:42:07.073 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/139 2023-07-08T13:42:07.073 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:07.075 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:07.476 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:07.477 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2023-07-08T13:42:07.762 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:07.766 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:07.767 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:07.768 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:07.768 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:08.148 INFO:teuthology.orchestra.run.smithi012.stdout:generic/140 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:08.152 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/140 2023-07-08T13:42:08.152 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/140 2023-07-08T13:42:08.153 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:08.154 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:08.584 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:08.585 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2023-07-08T13:42:08.827 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:08.831 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:08.832 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:08.833 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:08.833 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:09.436 INFO:teuthology.orchestra.run.smithi012.stdout:generic/141 0s 2023-07-08T13:42:09.441 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/141 2023-07-08T13:42:09.441 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:09.443 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:09.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:09.452 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2023-07-08T13:42:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:09.680 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:09.680 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:09.681 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:10.042 INFO:teuthology.orchestra.run.smithi012.stdout:generic/142 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:10.046 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/142 2023-07-08T13:42:10.046 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/142 2023-07-08T13:42:10.047 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:10.048 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:10.483 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:10.483 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2023-07-08T13:42:10.729 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:10.733 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:10.733 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:10.734 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:10.735 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:11.109 INFO:teuthology.orchestra.run.smithi012.stdout:generic/143 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:11.113 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/143 2023-07-08T13:42:11.114 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/143 2023-07-08T13:42:11.114 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:11.116 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:11.477 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:11.478 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2023-07-08T13:42:11.801 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:11.805 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:11.805 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:11.806 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:11.806 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:12.186 INFO:teuthology.orchestra.run.smithi012.stdout:generic/144 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:12.190 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/144 2023-07-08T13:42:12.190 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/144 2023-07-08T13:42:12.190 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:12.192 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:12.642 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:12.642 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2023-07-08T13:42:12.879 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:12.883 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:12.884 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:12.884 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:12.885 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:13.260 INFO:teuthology.orchestra.run.smithi012.stdout:generic/145 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:13.264 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/145 2023-07-08T13:42:13.264 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/145 2023-07-08T13:42:13.265 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:13.680 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:13.681 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2023-07-08T13:42:13.944 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:13.948 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:13.948 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:13.949 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:13.950 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:14.310 INFO:teuthology.orchestra.run.smithi012.stdout:generic/146 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:14.314 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/146 2023-07-08T13:42:14.314 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/146 2023-07-08T13:42:14.314 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:14.636 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:14.636 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2023-07-08T13:42:15.000 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:15.004 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:15.005 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:15.005 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:15.385 INFO:teuthology.orchestra.run.smithi012.stdout:generic/147 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:15.390 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/147 2023-07-08T13:42:15.390 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/147 2023-07-08T13:42:15.390 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:15.392 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:15.794 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:15.794 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2023-07-08T13:42:16.062 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:16.066 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:16.067 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:16.067 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:16.068 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:16.456 INFO:teuthology.orchestra.run.smithi012.stdout:generic/148 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/148 2023-07-08T13:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/148 2023-07-08T13:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:16.462 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:16.876 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:16.876 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2023-07-08T13:42:17.151 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:17.156 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:17.156 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:17.157 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:17.157 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:17.532 INFO:teuthology.orchestra.run.smithi012.stdout:generic/149 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:17.537 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/149 2023-07-08T13:42:17.537 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/149 2023-07-08T13:42:17.537 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:17.539 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:17.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:17.841 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2023-07-08T13:42:18.183 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:42:18.185 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:42:18.212 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:18.216 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:18.216 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:18.217 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:18.593 INFO:teuthology.orchestra.run.smithi012.stdout:generic/150 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:18.597 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/150 2023-07-08T13:42:18.597 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/150 2023-07-08T13:42:18.598 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:18.599 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:18.962 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:18.963 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2023-07-08T13:42:19.285 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:19.289 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:19.289 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:19.290 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:19.652 INFO:teuthology.orchestra.run.smithi012.stdout:generic/151 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:19.656 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/151 2023-07-08T13:42:19.657 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/151 2023-07-08T13:42:19.657 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:20.071 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:20.071 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2023-07-08T13:42:20.330 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:20.334 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:20.334 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:20.335 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:20.335 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:20.708 INFO:teuthology.orchestra.run.smithi012.stdout:generic/152 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:20.712 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/152 2023-07-08T13:42:20.712 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/152 2023-07-08T13:42:20.713 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:20.714 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:20.839 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:20.840 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2023-07-08T13:42:21.391 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:21.395 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:21.396 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:21.397 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:21.397 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:21.783 INFO:teuthology.orchestra.run.smithi012.stdout:generic/153 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:21.787 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/153 2023-07-08T13:42:21.787 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/153 2023-07-08T13:42:21.787 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:21.789 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:21.927 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:21.927 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2023-07-08T13:42:22.467 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:22.471 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:22.472 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:22.473 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:22.473 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:22.849 INFO:teuthology.orchestra.run.smithi012.stdout:generic/154 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:22.853 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/154 2023-07-08T13:42:22.853 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/154 2023-07-08T13:42:22.854 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:22.855 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:23.241 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:23.241 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2023-07-08T13:42:23.545 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:23.550 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:23.550 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:23.551 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:23.922 INFO:teuthology.orchestra.run.smithi012.stdout:generic/155 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/155 2023-07-08T13:42:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/155 2023-07-08T13:42:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:24.307 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:24.307 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2023-07-08T13:42:24.603 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:24.607 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:24.608 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:24.608 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:24.987 INFO:teuthology.orchestra.run.smithi012.stdout:generic/156 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:24.991 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/156 2023-07-08T13:42:24.991 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/156 2023-07-08T13:42:24.991 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:25.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:25.418 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2023-07-08T13:42:25.674 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:25.678 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:25.678 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:25.679 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:25.679 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:26.048 INFO:teuthology.orchestra.run.smithi012.stdout:generic/157 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:26.052 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/157 2023-07-08T13:42:26.053 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/157 2023-07-08T13:42:26.053 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:26.054 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:26.316 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:26.316 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2023-07-08T13:42:26.727 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:26.731 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:26.732 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:26.733 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:26.733 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:27.117 INFO:teuthology.orchestra.run.smithi012.stdout:generic/158 [not run] Dedupe not supported by test filesystem type: ceph 2023-07-08T13:42:27.122 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/158 2023-07-08T13:42:27.122 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/158 2023-07-08T13:42:27.122 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:27.124 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:27.378 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:27.378 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2023-07-08T13:42:27.776 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:27.780 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:27.781 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:27.782 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:27.782 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:28.167 INFO:teuthology.orchestra.run.smithi012.stdout:generic/159 [not run] file system doesn't support chattr +i 2023-07-08T13:42:28.172 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/159 2023-07-08T13:42:28.172 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/159 2023-07-08T13:42:28.172 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:28.174 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:28.203 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:28.203 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2023-07-08T13:42:28.425 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:28.430 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:28.796 INFO:teuthology.orchestra.run.smithi012.stdout:generic/160 [not run] file system doesn't support chattr +i 2023-07-08T13:42:28.800 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/160 2023-07-08T13:42:28.801 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/160 2023-07-08T13:42:28.801 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:28.802 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:28.840 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:28.840 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2023-07-08T13:42:29.063 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:29.067 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:29.068 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:29.069 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:29.069 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:29.757 INFO:teuthology.orchestra.run.smithi012.stdout:generic/161 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:30.022 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/161 2023-07-08T13:42:30.023 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/161 2023-07-08T13:42:30.023 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:30.028 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:30.059 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:30.060 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2023-07-08T13:42:30.293 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:30.297 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:30.297 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:30.298 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:30.299 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:30.848 INFO:teuthology.orchestra.run.smithi012.stdout:generic/162 [not run] Dedupe not supported by scratch filesystem type: ceph 2023-07-08T13:42:31.229 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/162 2023-07-08T13:42:31.229 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/162 2023-07-08T13:42:31.229 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:31.232 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:31.295 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:31.296 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2023-07-08T13:42:31.540 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:31.544 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:31.544 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:31.545 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:31.546 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:32.084 INFO:teuthology.orchestra.run.smithi012.stdout:generic/163 [not run] Dedupe not supported by scratch filesystem type: ceph 2023-07-08T13:42:32.533 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/163 2023-07-08T13:42:32.533 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/163 2023-07-08T13:42:32.533 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:32.535 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:32.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:32.564 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2023-07-08T13:42:32.786 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:32.790 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:32.791 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:32.792 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:32.792 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:33.760 INFO:teuthology.orchestra.run.smithi012.stdout:generic/164 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:33.780 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/164 2023-07-08T13:42:33.780 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/164 2023-07-08T13:42:33.780 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:33.789 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:33.823 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:33.824 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2023-07-08T13:42:34.046 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:34.050 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:34.051 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:34.052 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:34.052 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:34.967 INFO:teuthology.orchestra.run.smithi012.stdout:generic/165 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:34.974 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/165 2023-07-08T13:42:34.975 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/165 2023-07-08T13:42:34.975 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:35.019 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:35.054 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:35.054 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2023-07-08T13:42:35.283 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:35.287 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:35.288 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:35.289 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:35.289 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:36.156 INFO:teuthology.orchestra.run.smithi012.stdout:generic/166 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:36.238 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/166 2023-07-08T13:42:36.238 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/166 2023-07-08T13:42:36.239 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:36.291 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:36.291 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2023-07-08T13:42:36.520 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:36.524 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:36.524 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:36.525 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:36.526 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:37.493 INFO:teuthology.orchestra.run.smithi012.stdout:generic/167 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:37.520 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/167 2023-07-08T13:42:37.520 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/167 2023-07-08T13:42:37.521 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:37.525 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:37.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:37.561 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2023-07-08T13:42:37.786 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:37.791 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:37.791 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:37.792 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:38.728 INFO:teuthology.orchestra.run.smithi012.stdout:generic/168 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:38.769 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/168 2023-07-08T13:42:38.770 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/168 2023-07-08T13:42:38.770 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:38.773 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:38.802 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:38.802 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2023-07-08T13:42:39.027 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:39.031 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:39.031 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:39.032 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:39.033 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:39.848 INFO:teuthology.orchestra.run.smithi012.stdout:generic/169 0s 2023-07-08T13:42:39.853 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/169 2023-07-08T13:42:39.853 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:39.863 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:39.864 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2023-07-08T13:42:40.086 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:40.090 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:40.091 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:40.092 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:41.074 INFO:teuthology.orchestra.run.smithi012.stdout:generic/170 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:41.083 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/170 2023-07-08T13:42:41.084 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/170 2023-07-08T13:42:41.084 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:41.092 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:41.142 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:41.142 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2023-07-08T13:42:41.368 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:41.372 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:41.373 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:41.374 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:41.374 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:42.085 INFO:teuthology.orchestra.run.smithi012.stdout:generic/171 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:42.141 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/171 2023-07-08T13:42:42.141 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/171 2023-07-08T13:42:42.141 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:42.155 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:42.368 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:42.369 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2023-07-08T13:42:42.607 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:42.612 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:42.612 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:42.613 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:43.366 INFO:teuthology.orchestra.run.smithi012.stdout:generic/172 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:43.539 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/172 2023-07-08T13:42:43.539 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/172 2023-07-08T13:42:43.540 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:43.661 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:43.662 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2023-07-08T13:42:43.887 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:43.891 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:43.892 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:43.893 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:43.893 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:44.852 INFO:teuthology.orchestra.run.smithi012.stdout:generic/173 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:44.879 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/173 2023-07-08T13:42:44.879 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/173 2023-07-08T13:42:44.879 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:44.882 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:44.925 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:44.925 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2023-07-08T13:42:45.150 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:45.154 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:45.155 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:45.156 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:45.156 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:46.058 INFO:teuthology.orchestra.run.smithi012.stdout:generic/174 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:46.067 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/174 2023-07-08T13:42:46.068 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/174 2023-07-08T13:42:46.068 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:46.107 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:46.181 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:46.181 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2023-07-08T13:42:46.408 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:46.413 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:46.413 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:46.414 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:47.192 INFO:teuthology.orchestra.run.smithi012.stdout:generic/175 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:47.201 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/175 2023-07-08T13:42:47.201 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/175 2023-07-08T13:42:47.201 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:47.347 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:47.423 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:47.423 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2023-07-08T13:42:47.671 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:47.676 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:47.676 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:47.677 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:47.677 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:48.242 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:42:48.245 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:42:48.621 INFO:teuthology.orchestra.run.smithi012.stdout:generic/176 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:48.627 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/176 2023-07-08T13:42:48.627 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/176 2023-07-08T13:42:48.627 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:48.660 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:48.699 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:48.700 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2023-07-08T13:42:48.928 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:48.932 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:48.933 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:48.934 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:48.934 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:49.453 INFO:teuthology.orchestra.run.smithi012.stdout:generic/177 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-07-08T13:42:49.457 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/177 2023-07-08T13:42:49.457 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/177 2023-07-08T13:42:49.457 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:49.459 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:49.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:49.833 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2023-07-08T13:42:50.132 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:50.136 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:50.136 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:50.137 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:50.137 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:generic/178 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/178 2023-07-08T13:42:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/178 2023-07-08T13:42:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:50.924 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:50.925 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2023-07-08T13:42:51.210 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:51.214 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:51.214 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:51.215 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:51.216 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:51.592 INFO:teuthology.orchestra.run.smithi012.stdout:generic/179 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:51.596 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/179 2023-07-08T13:42:51.596 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/179 2023-07-08T13:42:51.596 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:51.598 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:51.807 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:51.807 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2023-07-08T13:42:52.269 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:52.273 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:52.274 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:52.275 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:52.275 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:52.651 INFO:teuthology.orchestra.run.smithi012.stdout:generic/180 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:52.655 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/180 2023-07-08T13:42:52.655 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/180 2023-07-08T13:42:52.655 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:53.060 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:53.060 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2023-07-08T13:42:53.342 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:53.346 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:53.347 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:53.348 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:53.348 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:53.727 INFO:teuthology.orchestra.run.smithi012.stdout:generic/181 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:42:53.731 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/181 2023-07-08T13:42:53.731 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/181 2023-07-08T13:42:53.731 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:53.733 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:53.899 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:53.900 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2023-07-08T13:42:54.417 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:54.421 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:54.422 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:54.423 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:54.423 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:54.795 INFO:teuthology.orchestra.run.smithi012.stdout:generic/182 [not run] Dedupe not supported by test filesystem type: ceph 2023-07-08T13:42:54.799 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/182 2023-07-08T13:42:54.799 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/182 2023-07-08T13:42:54.799 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:54.801 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:55.245 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:55.246 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2023-07-08T13:42:55.476 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:55.481 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:55.481 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:55.482 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:55.482 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:56.369 INFO:teuthology.orchestra.run.smithi012.stdout:generic/183 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:56.461 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/183 2023-07-08T13:42:56.461 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/183 2023-07-08T13:42:56.462 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:56.466 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:56.510 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:56.510 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2023-07-08T13:42:56.730 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:56.734 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:56.735 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:56.736 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:56.736 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:57.134 INFO:teuthology.orchestra.run.smithi012.stdout:generic/184 1s 2023-07-08T13:42:57.139 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/184 2023-07-08T13:42:57.140 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:57.141 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:57.167 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:57.168 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2023-07-08T13:42:57.394 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:57.398 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:57.399 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:57.400 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:57.400 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:58.310 INFO:teuthology.orchestra.run.smithi012.stdout:generic/185 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:58.327 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/185 2023-07-08T13:42:58.327 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/185 2023-07-08T13:42:58.327 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:58.328 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:58.418 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:58.419 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2023-07-08T13:42:58.645 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:58.649 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:58.650 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:58.651 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:59.536 INFO:teuthology.orchestra.run.smithi012.stdout:generic/186 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:42:59.627 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/186 2023-07-08T13:42:59.627 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/186 2023-07-08T13:42:59.627 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:42:59.633 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:42:59.658 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:42:59.658 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2023-07-08T13:42:59.879 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:42:59.883 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:42:59.884 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:42:59.884 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:42:59.885 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:00.746 INFO:teuthology.orchestra.run.smithi012.stdout:generic/187 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:00.837 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/187 2023-07-08T13:43:00.837 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/187 2023-07-08T13:43:00.837 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:00.857 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:00.911 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:00.911 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2023-07-08T13:43:01.134 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:01.138 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:01.138 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:01.139 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:01.140 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:02.078 INFO:teuthology.orchestra.run.smithi012.stdout:generic/188 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:02.102 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/188 2023-07-08T13:43:02.102 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/188 2023-07-08T13:43:02.102 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:02.107 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:02.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:02.136 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2023-07-08T13:43:02.362 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:02.367 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:02.367 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:02.368 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:02.368 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:03.263 INFO:teuthology.orchestra.run.smithi012.stdout:generic/189 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:03.343 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/189 2023-07-08T13:43:03.344 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/189 2023-07-08T13:43:03.344 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:03.344 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:03.378 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:03.378 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2023-07-08T13:43:03.608 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:03.612 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:03.612 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:03.613 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:03.614 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:04.404 INFO:teuthology.orchestra.run.smithi012.stdout:generic/190 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:04.411 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/190 2023-07-08T13:43:04.411 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/190 2023-07-08T13:43:04.411 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:04.415 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:04.645 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:04.646 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2023-07-08T13:43:04.871 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:04.875 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:04.875 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:04.876 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:04.877 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:generic/191 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:05.879 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/191 2023-07-08T13:43:05.880 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/191 2023-07-08T13:43:05.880 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:05.898 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:05.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:05.933 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2023-07-08T13:43:06.155 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:06.159 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:06.160 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:06.161 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:06.161 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:06.513 INFO:teuthology.orchestra.run.smithi012.stdout:generic/192 [not run] atime not maintained by ceph 2023-07-08T13:43:06.517 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/192 2023-07-08T13:43:06.518 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/192 2023-07-08T13:43:06.518 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:06.520 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:06.544 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:06.545 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2023-07-08T13:43:06.769 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:06.773 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:06.774 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:06.775 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:06.775 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:07.664 INFO:teuthology.orchestra.run.smithi012.stdout:generic/193 1s 2023-07-08T13:43:07.669 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/193 2023-07-08T13:43:07.669 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:07.671 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:07.898 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:07.899 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2023-07-08T13:43:08.344 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:08.348 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:08.349 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:08.350 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:09.045 INFO:teuthology.orchestra.run.smithi012.stdout:generic/194 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/194 2023-07-08T13:43:09.145 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/194 2023-07-08T13:43:09.146 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:09.365 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:09.365 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2023-07-08T13:43:09.598 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:09.603 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:09.603 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:09.604 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:09.604 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:10.517 INFO:teuthology.orchestra.run.smithi012.stdout:generic/195 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:10.567 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/195 2023-07-08T13:43:10.567 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/195 2023-07-08T13:43:10.568 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:10.582 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:10.614 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:10.614 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2023-07-08T13:43:10.836 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:10.841 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:10.842 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:10.842 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:11.586 INFO:teuthology.orchestra.run.smithi012.stdout:generic/196 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/196 2023-07-08T13:43:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/196 2023-07-08T13:43:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:11.599 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:11.888 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:11.888 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2023-07-08T13:43:12.110 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:12.114 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:12.114 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:12.115 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:12.116 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:13.036 INFO:teuthology.orchestra.run.smithi012.stdout:generic/197 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:13.051 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/197 2023-07-08T13:43:13.051 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/197 2023-07-08T13:43:13.051 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:13.054 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:13.115 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:13.115 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2023-07-08T13:43:13.341 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:13.346 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:13.346 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:13.347 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:13.347 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:14.087 INFO:teuthology.orchestra.run.smithi012.stdout:generic/198 1s 2023-07-08T13:43:14.092 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/198 2023-07-08T13:43:14.093 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:14.094 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:14.416 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:14.417 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2023-07-08T13:43:14.843 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:14.847 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:14.848 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:14.849 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:14.849 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:15.763 INFO:teuthology.orchestra.run.smithi012.stdout:generic/199 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:15.796 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/199 2023-07-08T13:43:15.796 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/199 2023-07-08T13:43:15.796 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:15.810 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:15.854 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:15.855 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2023-07-08T13:43:16.081 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:16.085 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:16.086 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:16.087 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:16.087 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:16.986 INFO:teuthology.orchestra.run.smithi012.stdout:generic/200 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:16.997 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/200 2023-07-08T13:43:16.997 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/200 2023-07-08T13:43:16.997 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:17.015 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:17.113 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:17.114 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2023-07-08T13:43:17.338 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:17.342 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:17.343 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:17.344 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:17.344 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:18.257 INFO:teuthology.orchestra.run.smithi012.stdout:generic/201 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:18.264 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/201 2023-07-08T13:43:18.264 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/201 2023-07-08T13:43:18.264 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:18.346 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:18.347 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2023-07-08T13:43:18.575 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:18.580 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:18.580 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:18.581 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:18.581 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:18.887 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:43:18.889 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:43:19.401 INFO:teuthology.orchestra.run.smithi012.stdout:generic/202 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:19.445 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/202 2023-07-08T13:43:19.445 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/202 2023-07-08T13:43:19.445 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:19.511 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:19.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:19.614 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2023-07-08T13:43:19.838 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:19.842 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:19.842 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:19.843 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:19.844 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:20.795 INFO:teuthology.orchestra.run.smithi012.stdout:generic/203 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:20.819 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/203 2023-07-08T13:43:20.819 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/203 2023-07-08T13:43:20.819 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:20.822 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:20.869 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:20.870 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2023-07-08T13:43:21.096 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:21.101 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:21.101 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:21.102 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:21.102 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:22.786 INFO:teuthology.orchestra.run.smithi012.stdout:generic/204 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T13:43:22.791 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/204 2023-07-08T13:43:22.792 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/204 2023-07-08T13:43:22.792 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:22.794 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:22.823 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:22.824 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2023-07-08T13:43:23.058 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:23.062 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:23.062 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:23.063 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:23.063 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:23.983 INFO:teuthology.orchestra.run.smithi012.stdout:generic/205 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:24.005 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/205 2023-07-08T13:43:24.005 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/205 2023-07-08T13:43:24.005 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:24.045 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:24.046 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2023-07-08T13:43:24.280 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:24.285 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:24.285 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:24.286 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:25.146 INFO:teuthology.orchestra.run.smithi012.stdout:generic/206 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:43:25.154 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/206 2023-07-08T13:43:25.154 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/206 2023-07-08T13:43:25.154 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:25.186 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:25.302 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:25.302 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2023-07-08T13:43:25.535 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:25.540 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:25.540 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:25.541 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:25.541 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:34.477 INFO:teuthology.orchestra.run.smithi012.stdout:generic/207 9s 2023-07-08T13:43:34.481 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/207 2023-07-08T13:43:34.482 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:43:34.483 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:34.837 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:43:34.838 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2023-07-08T13:43:35.135 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:43:35.139 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:43:35.139 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:43:35.140 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:43:35.141 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:43:48.947 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:43:48.950 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:44:19.253 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:44:19.256 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:44:49.318 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:44:49.320 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:45:19.380 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:45:19.382 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:45:49.443 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:45:49.446 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:46:19.514 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:46:19.517 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:46:49.596 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:46:49.599 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:46:55.638 INFO:teuthology.orchestra.run.smithi012.stdout:generic/208 200s 2023-07-08T13:46:55.686 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/208 2023-07-08T13:46:55.687 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:46:55.689 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:46:56.082 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:46:56.083 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2023-07-08T13:46:56.363 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:46:56.367 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:46:56.368 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:46:56.369 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:46:56.369 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:19.669 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:47:19.672 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:47:35.441 INFO:teuthology.orchestra.run.smithi012.stdout:generic/209 39s 2023-07-08T13:47:35.447 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/209 2023-07-08T13:47:35.447 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:35.450 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:35.853 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:35.854 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2023-07-08T13:47:36.135 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:36.139 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:36.140 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:36.141 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:36.141 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:36.602 INFO:teuthology.orchestra.run.smithi012.stdout:generic/210 0s 2023-07-08T13:47:36.607 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/210 2023-07-08T13:47:36.607 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:36.609 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:36.999 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:36.999 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2023-07-08T13:47:37.290 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:37.295 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:37.295 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:37.296 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:37.820 INFO:teuthology.orchestra.run.smithi012.stdout:generic/211 0s 2023-07-08T13:47:37.825 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/211 2023-07-08T13:47:37.825 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:37.860 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:37.861 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2023-07-08T13:47:38.081 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:38.086 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:38.086 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:38.087 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:38.087 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:38.533 INFO:teuthology.orchestra.run.smithi012.stdout:generic/212 0s 2023-07-08T13:47:38.537 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/212 2023-07-08T13:47:38.538 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:38.539 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:38.571 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:38.572 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2023-07-08T13:47:38.803 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:38.807 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:38.808 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:38.809 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:38.809 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:39.162 INFO:teuthology.orchestra.run.smithi012.stdout:generic/213 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T13:47:39.166 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/213 2023-07-08T13:47:39.167 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/213 2023-07-08T13:47:39.167 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:39.169 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:39.198 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:39.198 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2023-07-08T13:47:39.424 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:39.428 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:39.428 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:39.429 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:39.430 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:39.799 INFO:teuthology.orchestra.run.smithi012.stdout:generic/214 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T13:47:39.803 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/214 2023-07-08T13:47:39.804 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/214 2023-07-08T13:47:39.804 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:39.834 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:39.834 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2023-07-08T13:47:40.058 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:40.063 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:40.063 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:40.064 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:42.486 INFO:teuthology.orchestra.run.smithi012.stdout:generic/215 2s 2023-07-08T13:47:42.492 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/215 2023-07-08T13:47:42.492 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:42.551 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:42.551 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2023-07-08T13:47:42.784 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:42.788 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:42.789 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:42.790 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:43.761 INFO:teuthology.orchestra.run.smithi012.stdout:generic/216 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:47:43.776 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/216 2023-07-08T13:47:43.776 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/216 2023-07-08T13:47:43.776 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:43.780 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:43.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:43.811 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2023-07-08T13:47:44.035 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:44.040 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:44.040 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:44.041 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:44.041 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:44.913 INFO:teuthology.orchestra.run.smithi012.stdout:generic/217 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:47:45.039 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/217 2023-07-08T13:47:45.039 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/217 2023-07-08T13:47:45.039 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:45.070 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:45.071 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2023-07-08T13:47:45.295 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:45.299 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:45.300 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:45.301 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:46.033 INFO:teuthology.orchestra.run.smithi012.stdout:generic/218 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:47:46.052 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/218 2023-07-08T13:47:46.052 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/218 2023-07-08T13:47:46.052 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:46.243 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:46.324 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:46.325 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2023-07-08T13:47:46.549 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:46.553 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:46.554 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:46.555 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:46.555 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:46.908 INFO:teuthology.orchestra.run.smithi012.stdout:generic/219 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:47:46.912 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/219 2023-07-08T13:47:46.912 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/219 2023-07-08T13:47:46.912 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:46.941 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:46.942 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2023-07-08T13:47:47.163 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:47.168 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:47.168 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:47.169 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:47.169 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:48.098 INFO:teuthology.orchestra.run.smithi012.stdout:generic/220 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:47:48.159 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/220 2023-07-08T13:47:48.159 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/220 2023-07-08T13:47:48.159 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:48.164 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:48.198 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:48.199 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2023-07-08T13:47:48.421 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:48.426 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:48.426 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:48.427 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:48.427 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:49.736 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:47:49.739 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:47:49.798 INFO:teuthology.orchestra.run.smithi012.stdout:generic/221 1s 2023-07-08T13:47:49.804 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/221 2023-07-08T13:47:49.804 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:49.835 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:49.835 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2023-07-08T13:47:50.116 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:50.120 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:50.121 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:50.122 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:50.122 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:51.060 INFO:teuthology.orchestra.run.smithi012.stdout:generic/222 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:47:51.076 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/222 2023-07-08T13:47:51.076 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/222 2023-07-08T13:47:51.076 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:51.097 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:51.125 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:51.126 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2023-07-08T13:47:51.359 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:51.363 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:51.364 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:51.365 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:51.735 INFO:teuthology.orchestra.run.smithi012.stdout:generic/223 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T13:47:51.739 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/223 2023-07-08T13:47:51.739 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/223 2023-07-08T13:47:51.739 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:51.741 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:51.771 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:51.772 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2023-07-08T13:47:52.002 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:52.006 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:52.007 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:52.008 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:52.008 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:52.357 INFO:teuthology.orchestra.run.smithi012.stdout:generic/224 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T13:47:52.361 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/224 2023-07-08T13:47:52.362 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/224 2023-07-08T13:47:52.362 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:52.401 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:52.402 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2023-07-08T13:47:52.626 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:52.630 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:52.631 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:52.632 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:53.028 INFO:teuthology.orchestra.run.smithi012.stdout:generic/225 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-07-08T13:47:53.032 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/225 2023-07-08T13:47:53.032 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/225 2023-07-08T13:47:53.032 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:53.034 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:53.419 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:53.419 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2023-07-08T13:47:53.702 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:53.706 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:53.706 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:53.707 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:53.707 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:54.103 INFO:teuthology.orchestra.run.smithi012.stdout:generic/226 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T13:47:54.107 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/226 2023-07-08T13:47:54.107 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/226 2023-07-08T13:47:54.108 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:54.109 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:54.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:54.147 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2023-07-08T13:47:54.380 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:54.385 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:54.385 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:54.386 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:54.386 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:55.188 INFO:teuthology.orchestra.run.smithi012.stdout:generic/227 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:47:55.300 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/227 2023-07-08T13:47:55.301 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/227 2023-07-08T13:47:55.301 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:55.339 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:55.389 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:55.390 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2023-07-08T13:47:55.624 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:55.628 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:55.629 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:55.629 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:55.630 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:56.002 INFO:teuthology.orchestra.run.smithi012.stdout:generic/228 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T13:47:56.006 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/228 2023-07-08T13:47:56.006 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/228 2023-07-08T13:47:56.007 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:56.033 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:56.033 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2023-07-08T13:47:56.258 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:56.262 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:56.263 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:56.264 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:56.264 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:57.224 INFO:teuthology.orchestra.run.smithi012.stdout:generic/229 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:47:57.237 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/229 2023-07-08T13:47:57.237 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/229 2023-07-08T13:47:57.237 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:57.250 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:57.281 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:57.281 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2023-07-08T13:47:57.508 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:57.513 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:57.513 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:57.514 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:57.514 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:57.869 INFO:teuthology.orchestra.run.smithi012.stdout:generic/230 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:47:57.873 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/230 2023-07-08T13:47:57.873 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/230 2023-07-08T13:47:57.873 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:57.875 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:57.900 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:57.901 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2023-07-08T13:47:58.122 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:58.127 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:58.127 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:58.128 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:58.128 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:58.466 INFO:teuthology.orchestra.run.smithi012.stdout:generic/231 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:47:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/231 2023-07-08T13:47:58.471 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/231 2023-07-08T13:47:58.471 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:58.473 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:58.499 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:58.499 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2023-07-08T13:47:58.723 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:58.727 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:58.728 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:58.729 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:58.729 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:59.066 INFO:teuthology.orchestra.run.smithi012.stdout:generic/232 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:47:59.070 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/232 2023-07-08T13:47:59.070 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/232 2023-07-08T13:47:59.070 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:59.104 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:59.105 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2023-07-08T13:47:59.326 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:59.330 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:59.331 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:59.332 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:59.332 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:59.666 INFO:teuthology.orchestra.run.smithi012.stdout:generic/233 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:47:59.670 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/233 2023-07-08T13:47:59.670 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/233 2023-07-08T13:47:59.671 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:47:59.672 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:47:59.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:47:59.694 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2023-07-08T13:47:59.925 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:47:59.929 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:47:59.929 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:47:59.930 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:47:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:48:00.273 INFO:teuthology.orchestra.run.smithi012.stdout:generic/234 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:48:00.277 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/234 2023-07-08T13:48:00.277 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/234 2023-07-08T13:48:00.277 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:48:00.279 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:48:00.307 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:48:00.308 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2023-07-08T13:48:00.530 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:48:00.534 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:48:00.535 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:48:00.536 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:48:00.536 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:48:00.881 INFO:teuthology.orchestra.run.smithi012.stdout:generic/235 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:48:00.885 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/235 2023-07-08T13:48:00.885 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/235 2023-07-08T13:48:00.885 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:48:00.887 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:48:00.906 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:48:00.907 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2023-07-08T13:48:01.133 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:48:01.137 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:48:01.137 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:48:01.138 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:48:01.139 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:48:02.550 INFO:teuthology.orchestra.run.smithi012.stdout:generic/236 1s 2023-07-08T13:48:02.556 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/236 2023-07-08T13:48:02.556 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:48:02.558 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:48:02.979 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:48:02.980 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2023-07-08T13:48:03.234 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:48:03.238 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:48:03.239 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:48:03.240 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:48:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:48:03.687 INFO:teuthology.orchestra.run.smithi012.stdout:generic/237 0s 2023-07-08T13:48:03.692 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/237 2023-07-08T13:48:03.692 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:48:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:48:04.057 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:48:04.057 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2023-07-08T13:48:04.368 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:48:04.373 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:48:04.373 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:48:04.374 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:48:04.374 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:48:05.335 INFO:teuthology.orchestra.run.smithi012.stdout:generic/238 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:48:05.352 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/238 2023-07-08T13:48:05.355 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/238 2023-07-08T13:48:05.355 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:48:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:48:05.386 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:48:05.387 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2023-07-08T13:48:05.623 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:48:05.628 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:48:05.628 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:48:05.629 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:48:05.629 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:48:19.813 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:48:19.950 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:48:20.024 INFO:tasks.ceph.osd.6.smithi160.stderr:2023-07-08T13:48:20.023+0000 7fb3c0965700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 115591) UID: 0 2023-07-08T13:48:20.025 INFO:tasks.ceph.osd.5.smithi160.stderr:2023-07-08T13:48:20.023+0000 7f3d3601f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 115591) UID: 0 2023-07-08T13:48:20.025 INFO:tasks.ceph.osd.7.smithi160.stderr:2023-07-08T13:48:20.023+0000 7f0f5014f700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 115591) UID: 0 2023-07-08T13:48:20.026 INFO:tasks.ceph.osd.4.smithi160.stderr:2023-07-08T13:48:20.025+0000 7efd6f6e9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 115591) UID: 0 2023-07-08T13:50:08.389 INFO:teuthology.orchestra.run.smithi012.stdout:generic/239 123s 2023-07-08T13:50:08.394 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/239 2023-07-08T13:50:08.394 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:50:08.396 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:50:08.790 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:50:08.791 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2023-07-08T13:50:09.078 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:50:09.082 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:50:09.083 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:50:09.084 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:50:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:50:09.868 INFO:teuthology.orchestra.run.smithi012.stdout:generic/240 0s 2023-07-08T13:50:09.907 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/240 2023-07-08T13:50:09.907 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:50:09.909 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:50:10.272 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:50:10.273 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2023-07-08T13:50:10.632 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:50:10.637 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:50:10.637 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:50:10.638 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:50:10.638 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:24.017 INFO:teuthology.orchestra.run.smithi012.stdout:generic/241 74s 2023-07-08T13:51:24.022 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/241 2023-07-08T13:51:24.022 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:25.139 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:25.140 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2023-07-08T13:51:25.392 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:25.396 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:25.397 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:25.398 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:25.398 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:26.671 INFO:teuthology.orchestra.run.smithi012.stdout:generic/242 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:51:26.744 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/242 2023-07-08T13:51:26.744 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/242 2023-07-08T13:51:26.745 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:26.748 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:26.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:26.773 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2023-07-08T13:51:27.002 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:27.007 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:27.007 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:27.008 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:27.008 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:27.902 INFO:teuthology.orchestra.run.smithi012.stdout:generic/243 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:51:27.910 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/243 2023-07-08T13:51:27.910 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/243 2023-07-08T13:51:27.911 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:27.932 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:27.997 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:27.997 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2023-07-08T13:51:28.226 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:28.230 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:28.231 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:28.232 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:28.232 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:28.567 INFO:teuthology.orchestra.run.smithi012.stdout:generic/244 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:51:28.571 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/244 2023-07-08T13:51:28.571 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/244 2023-07-08T13:51:28.571 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:28.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:28.599 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2023-07-08T13:51:28.826 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:28.831 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:28.831 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:28.832 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:28.832 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:29.269 INFO:teuthology.orchestra.run.smithi012.stdout:generic/245 0s 2023-07-08T13:51:29.319 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/245 2023-07-08T13:51:29.319 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:29.321 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:29.673 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:29.674 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2023-07-08T13:51:29.996 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:30.001 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:30.001 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:30.002 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:30.002 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:30.467 INFO:teuthology.orchestra.run.smithi012.stdout:generic/246 0s 2023-07-08T13:51:30.544 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/246 2023-07-08T13:51:30.544 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:30.956 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:30.957 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2023-07-08T13:51:31.227 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:31.231 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:31.232 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:31.233 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:31.233 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:40.892 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:51:40.895 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:51:40.947 INFO:tasks.ceph.mds.a.smithi012.stderr:2023-07-08T13:51:40.945+0000 7f47874c4700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 353864) UID: 0 2023-07-08T13:51:40.947 INFO:tasks.ceph.mds.c.smithi012.stderr:2023-07-08T13:51:40.945+0000 7f187cc06700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 353864) UID: 0 2023-07-08T13:51:41.996 INFO:teuthology.orchestra.run.smithi012.stdout:generic/247 10s 2023-07-08T13:51:42.001 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/247 2023-07-08T13:51:42.001 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:42.003 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:42.011 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:42.012 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2023-07-08T13:51:42.238 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:42.242 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:42.242 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:42.243 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:42.631 INFO:teuthology.orchestra.run.smithi012.stdout:generic/248 0s 2023-07-08T13:51:42.636 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/248 2023-07-08T13:51:42.636 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:42.667 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:42.668 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2023-07-08T13:51:42.889 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:42.893 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:42.894 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:42.895 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:43.518 INFO:teuthology.orchestra.run.smithi012.stdout:generic/249 0s 2023-07-08T13:51:43.523 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/249 2023-07-08T13:51:43.523 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:43.789 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:43.790 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2023-07-08T13:51:44.218 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:44.222 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:44.222 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:44.223 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:44.224 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:44.726 INFO:teuthology.orchestra.run.smithi012.stdout:generic/250 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:51:44.729 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/250 2023-07-08T13:51:44.730 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/250 2023-07-08T13:51:44.730 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:44.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:44.756 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2023-07-08T13:51:44.987 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:44.992 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:44.992 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:44.993 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:45.501 INFO:teuthology.orchestra.run.smithi012.stdout:generic/251 [not run] FITRIM not supported on /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:45.518 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/251 2023-07-08T13:51:45.518 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/251 2023-07-08T13:51:45.518 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:45.520 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:45.548 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:45.549 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2023-07-08T13:51:45.777 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:45.781 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:45.782 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:45.783 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:45.783 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:46.130 INFO:teuthology.orchestra.run.smithi012.stdout:generic/252 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:51:46.134 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/252 2023-07-08T13:51:46.134 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/252 2023-07-08T13:51:46.134 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:46.136 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:46.156 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:46.157 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2023-07-08T13:51:46.385 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:46.389 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:46.389 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:46.390 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:46.390 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:47.355 INFO:teuthology.orchestra.run.smithi012.stdout:generic/253 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:51:47.378 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/253 2023-07-08T13:51:47.378 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/253 2023-07-08T13:51:47.379 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:47.388 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:47.423 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:47.423 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2023-07-08T13:51:47.653 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:47.657 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:47.657 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:47.658 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:47.659 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:48.372 INFO:teuthology.orchestra.run.smithi012.stdout:generic/254 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:51:48.551 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/254 2023-07-08T13:51:48.552 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/254 2023-07-08T13:51:48.552 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:48.556 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:48.680 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:48.680 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2023-07-08T13:51:48.908 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:48.912 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:48.913 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:48.914 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:48.914 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:49.364 INFO:teuthology.orchestra.run.smithi012.stdout:generic/255 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T13:51:49.368 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/255 2023-07-08T13:51:49.369 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/255 2023-07-08T13:51:49.369 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:49.370 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:49.499 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:49.500 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2023-07-08T13:51:50.055 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:50.059 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:50.059 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:50.060 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:50.061 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:50.809 INFO:teuthology.orchestra.run.smithi012.stdout:generic/256 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T13:51:50.813 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/256 2023-07-08T13:51:50.813 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/256 2023-07-08T13:51:50.814 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:50.816 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:50.842 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:50.842 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2023-07-08T13:51:51.072 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:51.076 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:51.076 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:51.077 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:51.077 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:51.995 INFO:teuthology.orchestra.run.smithi012.stdout:generic/257 0s 2023-07-08T13:51:51.999 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/257 2023-07-08T13:51:52.000 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:52.002 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:52.537 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:52.537 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2023-07-08T13:51:52.962 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:52.966 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:52.967 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:52.968 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:52.968 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:53.304 INFO:teuthology.orchestra.run.smithi012.stdout:generic/258 [not run] ceph does not support negative timestamps 2023-07-08T13:51:53.308 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/258 2023-07-08T13:51:53.308 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/258 2023-07-08T13:51:53.308 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:53.310 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:53.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:53.333 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2023-07-08T13:51:53.558 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:53.563 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:53.563 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:53.564 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:53.564 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:54.308 INFO:teuthology.orchestra.run.smithi012.stdout:generic/259 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:51:54.521 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/259 2023-07-08T13:51:54.521 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/259 2023-07-08T13:51:54.521 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:54.524 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:54.553 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:54.553 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2023-07-08T13:51:54.787 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:54.791 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:54.792 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:54.793 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:54.793 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:55.333 INFO:teuthology.orchestra.run.smithi012.stdout:generic/260 [not run] FITRIM not supported on /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:55.359 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/260 2023-07-08T13:51:55.359 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/260 2023-07-08T13:51:55.359 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:55.361 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:55.393 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:55.394 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2023-07-08T13:51:55.632 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:55.637 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:55.637 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:55.638 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:55.638 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:56.461 INFO:teuthology.orchestra.run.smithi012.stdout:generic/261 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:51:56.589 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/261 2023-07-08T13:51:56.589 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/261 2023-07-08T13:51:56.589 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:56.615 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:56.667 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:56.667 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2023-07-08T13:51:56.894 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:56.898 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:56.898 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:56.899 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:56.900 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:57.874 INFO:teuthology.orchestra.run.smithi012.stdout:generic/262 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:51:57.886 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/262 2023-07-08T13:51:57.887 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/262 2023-07-08T13:51:57.887 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:51:57.889 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:51:57.925 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:51:57.926 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2023-07-08T13:51:58.164 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:51:58.168 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:51:58.168 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:51:58.169 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:51:58.170 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:39.721 INFO:teuthology.orchestra.run.smithi012.stdout:generic/263 41s 2023-07-08T13:52:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/263 2023-07-08T13:52:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:39.741 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:39.975 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:39.976 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2023-07-08T13:52:40.461 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:40.465 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:40.465 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:40.466 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:41.252 INFO:teuthology.orchestra.run.smithi012.stdout:generic/264 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:52:41.420 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/264 2023-07-08T13:52:41.420 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/264 2023-07-08T13:52:41.420 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:41.470 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:41.512 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:41.513 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2023-07-08T13:52:41.737 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:41.742 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:41.742 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:41.744 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:42.636 INFO:teuthology.orchestra.run.smithi012.stdout:generic/265 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:52:42.697 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/265 2023-07-08T13:52:42.697 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/265 2023-07-08T13:52:42.697 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:42.749 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:42.750 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2023-07-08T13:52:42.992 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:42.996 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:42.997 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:42.998 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:generic/266 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:52:44.007 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/266 2023-07-08T13:52:44.007 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/266 2023-07-08T13:52:44.007 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:44.034 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:44.092 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:44.092 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2023-07-08T13:52:44.319 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:44.323 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:44.324 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:44.325 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:44.325 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:45.279 INFO:teuthology.orchestra.run.smithi012.stdout:generic/267 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:52:45.286 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/267 2023-07-08T13:52:45.286 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/267 2023-07-08T13:52:45.287 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:45.375 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:45.423 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:45.424 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2023-07-08T13:52:45.659 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:45.663 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:45.664 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:45.665 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:46.582 INFO:teuthology.orchestra.run.smithi012.stdout:generic/268 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:52:46.618 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/268 2023-07-08T13:52:46.618 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/268 2023-07-08T13:52:46.618 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:46.631 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:46.851 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:46.852 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2023-07-08T13:52:47.090 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:47.094 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:47.095 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:47.096 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:47.096 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:47.428 INFO:teuthology.orchestra.run.smithi012.stdout:generic/269 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T13:52:47.432 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/269 2023-07-08T13:52:47.432 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/269 2023-07-08T13:52:47.432 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:47.434 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:47.458 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:47.459 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2023-07-08T13:52:47.685 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:47.689 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:47.690 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:47.691 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:47.691 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:48.024 INFO:teuthology.orchestra.run.smithi012.stdout:generic/270 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:52:48.027 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/270 2023-07-08T13:52:48.028 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/270 2023-07-08T13:52:48.028 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:48.030 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:48.061 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:48.061 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2023-07-08T13:52:48.291 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:48.295 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:48.295 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:48.296 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:48.296 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:49.167 INFO:teuthology.orchestra.run.smithi012.stdout:generic/271 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:52:49.227 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/271 2023-07-08T13:52:49.227 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/271 2023-07-08T13:52:49.227 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:49.287 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:49.329 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:49.329 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2023-07-08T13:52:49.576 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:49.580 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:49.581 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:49.582 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:49.582 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:generic/272 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:52:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/272 2023-07-08T13:52:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/272 2023-07-08T13:52:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:50.608 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:50.629 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:50.629 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2023-07-08T13:52:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:50.857 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:50.858 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:50.859 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:51.215 INFO:teuthology.orchestra.run.smithi012.stdout:generic/273 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T13:52:51.219 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/273 2023-07-08T13:52:51.219 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/273 2023-07-08T13:52:51.220 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:51.221 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:51.241 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:51.242 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2023-07-08T13:52:51.473 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:51.479 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:51.825 INFO:teuthology.orchestra.run.smithi012.stdout:generic/274 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-07-08T13:52:51.829 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/274 2023-07-08T13:52:51.830 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/274 2023-07-08T13:52:51.830 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:51.832 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:51.860 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:51.860 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2023-07-08T13:52:52.084 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:52.088 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:52.088 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:52.089 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:52.089 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:52.431 INFO:teuthology.orchestra.run.smithi012.stdout:generic/275 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T13:52:52.435 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/275 2023-07-08T13:52:52.436 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/275 2023-07-08T13:52:52.436 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:52.438 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:52.460 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:52.461 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2023-07-08T13:52:52.686 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:52.690 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:52.691 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:52.692 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:52.692 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:53.661 INFO:teuthology.orchestra.run.smithi012.stdout:generic/276 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:52:53.704 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/276 2023-07-08T13:52:53.704 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/276 2023-07-08T13:52:53.705 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:53.707 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:53.735 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:53.735 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2023-07-08T13:52:53.964 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:53.969 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:53.969 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:53.970 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:53.970 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:54.583 INFO:teuthology.orchestra.run.smithi012.stdout:generic/277 [not run] file system doesn't support chattr +A 2023-07-08T13:52:54.587 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/277 2023-07-08T13:52:54.588 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/277 2023-07-08T13:52:54.588 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:54.590 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:54.617 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:54.617 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2023-07-08T13:52:54.838 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:54.842 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:54.843 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:54.844 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:54.844 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:55.787 INFO:teuthology.orchestra.run.smithi012.stdout:generic/278 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:52:55.862 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/278 2023-07-08T13:52:55.863 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/278 2023-07-08T13:52:55.863 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:55.865 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:55.892 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:55.893 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2023-07-08T13:52:56.115 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:56.119 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:56.119 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:56.120 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:56.120 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:57.077 INFO:teuthology.orchestra.run.smithi012.stdout:generic/279 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:52:57.086 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/279 2023-07-08T13:52:57.086 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/279 2023-07-08T13:52:57.086 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:57.098 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:57.154 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:57.154 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2023-07-08T13:52:57.377 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:57.381 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:57.381 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:57.383 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:57.383 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:57.738 INFO:teuthology.orchestra.run.smithi012.stdout:generic/280 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:52:57.742 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/280 2023-07-08T13:52:57.742 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/280 2023-07-08T13:52:57.742 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:57.744 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:57.770 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:57.770 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2023-07-08T13:52:57.997 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:58.001 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:58.001 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:58.002 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:58.002 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:58.852 INFO:teuthology.orchestra.run.smithi012.stdout:generic/281 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:52:58.980 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/281 2023-07-08T13:52:58.981 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/281 2023-07-08T13:52:58.981 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:52:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:52:59.059 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:52:59.060 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2023-07-08T13:52:59.479 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:52:59.482 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:52:59.483 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:52:59.484 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:52:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:00.459 INFO:teuthology.orchestra.run.smithi012.stdout:generic/282 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:00.514 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/282 2023-07-08T13:53:00.514 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/282 2023-07-08T13:53:00.514 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:00.517 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:00.550 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:00.550 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2023-07-08T13:53:00.783 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:00.787 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:00.788 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:00.788 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:01.751 INFO:teuthology.orchestra.run.smithi012.stdout:generic/283 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:01.771 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/283 2023-07-08T13:53:01.771 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/283 2023-07-08T13:53:01.771 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:01.780 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:01.831 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:01.831 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2023-07-08T13:53:02.066 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:02.071 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:02.071 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:02.072 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:02.072 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:02.880 INFO:teuthology.orchestra.run.smithi012.stdout:generic/284 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:03.041 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/284 2023-07-08T13:53:03.041 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/284 2023-07-08T13:53:03.042 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:03.084 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:03.108 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:03.109 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2023-07-08T13:53:03.330 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:03.334 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:03.334 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:03.335 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:03.336 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:04.787 INFO:teuthology.orchestra.run.smithi012.stdout:generic/285 1s 2023-07-08T13:53:04.792 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/285 2023-07-08T13:53:04.792 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:04.794 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:05.216 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:05.216 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2023-07-08T13:53:06.053 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:06.070 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:06.071 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:06.072 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:06.072 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:06.671 INFO:teuthology.orchestra.run.smithi012.stdout:generic/286 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T13:53:06.677 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/286 2023-07-08T13:53:06.677 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/286 2023-07-08T13:53:06.678 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:06.680 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:06.731 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:06.732 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2023-07-08T13:53:07.339 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:07.370 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:07.373 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:07.374 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:07.374 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:08.736 INFO:teuthology.orchestra.run.smithi012.stdout:generic/287 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:08.792 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/287 2023-07-08T13:53:08.792 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/287 2023-07-08T13:53:08.792 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:08.809 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:08.878 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:08.879 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2023-07-08T13:53:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:09.392 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:09.393 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:09.394 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:09.394 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:10.333 INFO:teuthology.orchestra.run.smithi012.stdout:generic/288 [not run] FITRIM not supported on /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:10.373 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/288 2023-07-08T13:53:10.373 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/288 2023-07-08T13:53:10.374 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:10.386 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:10.412 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:10.413 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2023-07-08T13:53:10.950 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:10.961 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:10.961 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:10.963 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:10.963 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:12.070 INFO:teuthology.orchestra.run.smithi012.stdout:generic/289 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:12.077 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/289 2023-07-08T13:53:12.077 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/289 2023-07-08T13:53:12.077 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:12.110 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:12.195 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:12.196 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2023-07-08T13:53:12.422 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:12.426 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:12.427 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:12.428 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:12.428 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:13.339 INFO:teuthology.orchestra.run.smithi012.stdout:generic/290 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:13.363 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/290 2023-07-08T13:53:13.363 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/290 2023-07-08T13:53:13.363 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:13.366 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:13.451 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:13.451 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2023-07-08T13:53:13.682 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:13.686 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:13.687 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:13.688 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:13.688 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:14.650 INFO:teuthology.orchestra.run.smithi012.stdout:generic/291 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:14.709 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/291 2023-07-08T13:53:14.709 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/291 2023-07-08T13:53:14.709 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:14.742 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:14.743 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2023-07-08T13:53:14.970 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:14.974 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:14.975 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:14.976 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:14.976 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:15.807 INFO:teuthology.orchestra.run.smithi012.stdout:generic/292 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/292 2023-07-08T13:53:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/292 2023-07-08T13:53:15.893 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:16.078 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:16.078 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2023-07-08T13:53:16.311 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:16.315 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:16.315 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:16.316 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:16.316 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:17.138 INFO:teuthology.orchestra.run.smithi012.stdout:generic/293 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:17.153 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/293 2023-07-08T13:53:17.154 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/293 2023-07-08T13:53:17.154 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:17.157 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:17.310 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:17.311 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2023-07-08T13:53:17.662 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:17.666 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:17.666 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:17.667 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:18.814 INFO:teuthology.orchestra.run.smithi012.stdout:generic/294 1s 2023-07-08T13:53:18.821 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/294 2023-07-08T13:53:18.822 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:18.823 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:18.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:18.833 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2023-07-08T13:53:19.065 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:19.069 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:19.070 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:19.071 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:19.071 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:20.045 INFO:teuthology.orchestra.run.smithi012.stdout:generic/295 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:20.056 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/295 2023-07-08T13:53:20.056 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/295 2023-07-08T13:53:20.056 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:20.070 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:20.103 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:20.104 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2023-07-08T13:53:20.332 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:20.336 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:20.337 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:20.338 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:20.338 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:21.098 INFO:teuthology.orchestra.run.smithi012.stdout:generic/296 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:21.177 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/296 2023-07-08T13:53:21.177 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/296 2023-07-08T13:53:21.177 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:21.326 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:21.377 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:21.378 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2023-07-08T13:53:21.614 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:21.618 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:21.619 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:21.620 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:22.308 INFO:teuthology.orchestra.run.smithi012.stdout:generic/297 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:22.314 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/297 2023-07-08T13:53:22.314 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/297 2023-07-08T13:53:22.314 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:22.317 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:22.583 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:22.583 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2023-07-08T13:53:22.842 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:22.846 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:22.846 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:22.847 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:22.848 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:23.753 INFO:teuthology.orchestra.run.smithi012.stdout:generic/298 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/298 2023-07-08T13:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/298 2023-07-08T13:53:23.760 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:23.854 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:23.855 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2023-07-08T13:53:24.084 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:24.088 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:24.089 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:24.513 INFO:teuthology.orchestra.run.smithi012.stdout:generic/299 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:53:24.516 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/299 2023-07-08T13:53:24.517 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/299 2023-07-08T13:53:24.517 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:24.549 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:24.550 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2023-07-08T13:53:24.781 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:24.785 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:24.785 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:24.786 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:24.787 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:25.176 INFO:teuthology.orchestra.run.smithi012.stdout:generic/300 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:53:25.180 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/300 2023-07-08T13:53:25.180 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/300 2023-07-08T13:53:25.180 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:25.182 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:25.212 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:25.212 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2023-07-08T13:53:25.441 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:25.445 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:25.445 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:25.446 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:25.446 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:26.338 INFO:teuthology.orchestra.run.smithi012.stdout:generic/301 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:26.345 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/301 2023-07-08T13:53:26.345 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/301 2023-07-08T13:53:26.346 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:26.447 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:26.448 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2023-07-08T13:53:26.705 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:26.709 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:26.710 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:26.710 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:26.711 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:27.663 INFO:teuthology.orchestra.run.smithi012.stdout:generic/302 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:27.730 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/302 2023-07-08T13:53:27.730 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/302 2023-07-08T13:53:27.730 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:27.731 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:27.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:27.756 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2023-07-08T13:53:27.981 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:27.985 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:27.985 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:27.986 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:28.371 INFO:teuthology.orchestra.run.smithi012.stdout:generic/303 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:53:28.375 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/303 2023-07-08T13:53:28.375 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/303 2023-07-08T13:53:28.375 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:28.377 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:28.850 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:28.850 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2023-07-08T13:53:29.084 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:29.088 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:29.088 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:29.089 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:29.090 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:29.471 INFO:teuthology.orchestra.run.smithi012.stdout:generic/304 [not run] Dedupe not supported by test filesystem type: ceph 2023-07-08T13:53:29.475 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/304 2023-07-08T13:53:29.475 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/304 2023-07-08T13:53:29.475 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:29.477 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:29.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:29.932 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2023-07-08T13:53:30.204 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:30.208 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:30.208 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:30.209 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:30.209 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:31.081 INFO:teuthology.orchestra.run.smithi012.stdout:generic/305 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:53:31.091 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/305 2023-07-08T13:53:31.091 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/305 2023-07-08T13:53:31.091 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:31.095 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:31.179 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:31.179 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2023-07-08T13:53:31.413 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:31.418 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:31.418 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:31.419 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:32.631 INFO:teuthology.orchestra.run.smithi012.stdout:generic/306 1s 2023-07-08T13:53:32.646 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/306 2023-07-08T13:53:32.647 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:32.649 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:33.080 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:33.081 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2023-07-08T13:53:33.335 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:33.339 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:33.339 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:33.340 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:35.246 INFO:teuthology.orchestra.run.smithi012.stdout:generic/307 2s 2023-07-08T13:53:35.251 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/307 2023-07-08T13:53:35.251 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:35.253 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:35.261 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:35.261 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2023-07-08T13:53:35.489 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:35.493 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:35.493 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:35.494 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:35.495 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:generic/308 0s 2023-07-08T13:53:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/308 2023-07-08T13:53:35.873 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:35.875 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:35.905 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:35.906 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2023-07-08T13:53:36.130 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:36.134 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:36.134 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:36.135 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:36.136 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:37.537 INFO:teuthology.orchestra.run.smithi012.stdout:generic/309 1s 2023-07-08T13:53:37.542 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/309 2023-07-08T13:53:37.542 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:53:37.544 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:53:37.991 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:53:37.991 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2023-07-08T13:53:38.242 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:53:38.246 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:53:38.246 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:53:38.247 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:53:38.248 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:54:54.760 INFO:teuthology.orchestra.run.smithi012.stdout:generic/310 76s 2023-07-08T13:54:54.765 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/310 2023-07-08T13:54:54.765 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:54:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:54:55.203 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:54:55.203 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2023-07-08T13:54:55.684 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:54:55.688 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:54:55.689 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:54:55.690 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:54:55.690 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:54:56.104 INFO:teuthology.orchestra.run.smithi012.stdout:generic/311 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:54:56.108 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/311 2023-07-08T13:54:56.108 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/311 2023-07-08T13:54:56.108 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:54:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:54:56.134 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:54:56.135 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2023-07-08T13:54:56.358 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:54:56.362 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:54:56.363 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:54:56.364 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:54:56.364 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:54:56.714 INFO:teuthology.orchestra.run.smithi012.stdout:generic/312 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T13:54:56.718 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/312 2023-07-08T13:54:56.719 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/312 2023-07-08T13:54:56.719 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:54:56.721 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:54:56.746 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:54:56.746 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2023-07-08T13:54:56.968 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:54:56.972 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:54:56.973 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:54:56.974 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:54:56.974 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:01.357 INFO:teuthology.orchestra.run.smithi012.stdout:generic/313 4s 2023-07-08T13:55:01.363 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/313 2023-07-08T13:55:01.363 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:55:01.365 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:01.396 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:55:01.397 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2023-07-08T13:55:01.631 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:55:01.635 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:55:01.635 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:55:01.636 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:55:01.637 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:02.235 INFO:teuthology.orchestra.run.smithi012.stdout:generic/314 1s 2023-07-08T13:55:02.241 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/314 2023-07-08T13:55:02.241 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:55:02.242 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:02.274 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:55:02.275 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2023-07-08T13:55:02.496 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:55:02.500 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:55:02.501 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:55:02.502 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:55:02.502 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:02.841 INFO:teuthology.orchestra.run.smithi012.stdout:generic/315 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-07-08T13:55:02.845 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/315 2023-07-08T13:55:02.845 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/315 2023-07-08T13:55:02.845 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:55:02.847 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:02.873 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:55:02.874 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2023-07-08T13:55:03.097 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:55:03.101 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:55:03.101 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:55:03.102 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:55:03.102 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:03.571 INFO:teuthology.orchestra.run.smithi012.stdout:generic/316 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-07-08T13:55:03.575 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/316 2023-07-08T13:55:03.576 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/316 2023-07-08T13:55:03.576 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:55:03.577 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:03.936 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:55:03.937 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2023-07-08T13:55:04.222 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:55:04.226 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:55:04.227 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:55:04.228 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:55:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:08.773 INFO:teuthology.orchestra.run.smithi012.stdout:generic/317 4s 2023-07-08T13:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/317 2023-07-08T13:55:08.778 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:55:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:08.789 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:55:08.789 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2023-07-08T13:55:09.015 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:55:09.019 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:55:09.020 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:55:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:09.885 INFO:teuthology.orchestra.run.smithi012.stdout:generic/318 0s 2023-07-08T13:55:09.890 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/318 2023-07-08T13:55:09.890 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:55:09.892 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:09.900 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:55:09.901 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2023-07-08T13:55:10.122 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:55:10.126 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:55:10.126 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:55:10.127 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:55:10.128 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:10.726 INFO:teuthology.orchestra.run.smithi012.stdout:generic/319 0s 2023-07-08T13:55:10.731 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/319 2023-07-08T13:55:10.731 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:55:10.733 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:10.741 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:55:10.742 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2023-07-08T13:55:10.968 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:55:10.972 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:55:10.973 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:55:10.974 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:55:10.974 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:11.320 INFO:teuthology.orchestra.run.smithi012.stdout:generic/320 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T13:55:11.324 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/320 2023-07-08T13:55:11.324 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/320 2023-07-08T13:55:11.324 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:55:11.326 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:11.350 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:55:11.350 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2023-07-08T13:55:11.588 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:55:11.592 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:55:11.593 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:55:11.594 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:55:11.594 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:11.960 INFO:teuthology.orchestra.run.smithi012.stdout:generic/321 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:55:11.964 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/321 2023-07-08T13:55:11.964 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/321 2023-07-08T13:55:11.964 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:55:11.966 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:11.990 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:55:11.990 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2023-07-08T13:55:12.212 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:55:12.216 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:55:12.216 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:55:12.217 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:55:12.218 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:12.558 INFO:teuthology.orchestra.run.smithi012.stdout:generic/322 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:55:12.562 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/322 2023-07-08T13:55:12.562 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/322 2023-07-08T13:55:12.562 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:55:12.564 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:12.584 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:55:12.584 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2023-07-08T13:55:12.805 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:55:12.809 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:55:12.810 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:55:12.811 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:55:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:55:17.806 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:55:17.808 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:55:47.887 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:55:47.890 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:56:18.144 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:56:18.146 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:56:48.455 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:56:48.457 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:56:48.500 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-07-08T13:56:48.498+0000 7f83fa332700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 396646) UID: 0 2023-07-08T13:56:48.500 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-07-08T13:56:48.498+0000 7fe6c1b3c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 396646) UID: 0 2023-07-08T13:56:48.500 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-07-08T13:56:48.498+0000 7fcf305dc700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 396646) UID: 0 2023-07-08T13:56:48.501 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-07-08T13:56:48.498+0000 7f589b052700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 396646) UID: 0 2023-07-08T13:57:13.724 INFO:teuthology.orchestra.run.smithi012.stdout:generic/323 120s 2023-07-08T13:57:13.739 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/323 2023-07-08T13:57:13.739 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:14.101 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:14.102 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2023-07-08T13:57:14.404 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:14.409 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:14.409 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:14.410 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:14.743 INFO:teuthology.orchestra.run.smithi012.stdout:generic/324 [not run] defragmentation not supported for fstype "ceph" 2023-07-08T13:57:14.748 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/324 2023-07-08T13:57:14.748 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/324 2023-07-08T13:57:14.748 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:14.750 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:14.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:14.774 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2023-07-08T13:57:14.998 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:15.002 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:15.002 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:15.003 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:15.350 INFO:teuthology.orchestra.run.smithi012.stdout:generic/325 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:57:15.354 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/325 2023-07-08T13:57:15.354 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/325 2023-07-08T13:57:15.354 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:15.356 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:15.378 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:15.378 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2023-07-08T13:57:15.607 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:15.611 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:15.612 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:15.613 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:15.613 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:16.588 INFO:teuthology.orchestra.run.smithi012.stdout:generic/326 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:57:16.594 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/326 2023-07-08T13:57:16.595 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/326 2023-07-08T13:57:16.595 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:16.607 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:16.638 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:16.639 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2023-07-08T13:57:16.881 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:16.885 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:16.886 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:16.887 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:16.887 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:17.769 INFO:teuthology.orchestra.run.smithi012.stdout:generic/327 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:57:17.866 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/327 2023-07-08T13:57:17.866 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/327 2023-07-08T13:57:17.866 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:17.874 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:17.900 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:17.900 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2023-07-08T13:57:18.121 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:18.125 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:18.125 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:18.126 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:18.126 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:19.091 INFO:teuthology.orchestra.run.smithi012.stdout:generic/328 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:57:19.129 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/328 2023-07-08T13:57:19.129 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/328 2023-07-08T13:57:19.129 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:19.132 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:19.155 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:19.155 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2023-07-08T13:57:19.384 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:19.388 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:19.389 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:19.390 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:19.390 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:20.342 INFO:teuthology.orchestra.run.smithi012.stdout:generic/329 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:57:20.415 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/329 2023-07-08T13:57:20.415 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/329 2023-07-08T13:57:20.415 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:20.426 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:20.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:20.455 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2023-07-08T13:57:20.686 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:20.690 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:20.690 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:20.691 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:20.691 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:21.424 INFO:teuthology.orchestra.run.smithi012.stdout:generic/330 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:57:21.431 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/330 2023-07-08T13:57:21.431 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/330 2023-07-08T13:57:21.431 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:21.611 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:21.688 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:21.688 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2023-07-08T13:57:21.925 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:21.929 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:21.930 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:21.931 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:21.931 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:22.872 INFO:teuthology.orchestra.run.smithi012.stdout:generic/331 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:57:22.881 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/331 2023-07-08T13:57:22.881 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/331 2023-07-08T13:57:22.881 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:22.909 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:22.948 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:22.949 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2023-07-08T13:57:23.195 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:23.199 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:23.199 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:23.200 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:24.060 INFO:teuthology.orchestra.run.smithi012.stdout:generic/332 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:57:24.167 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/332 2023-07-08T13:57:24.167 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/332 2023-07-08T13:57:24.167 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:24.206 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:24.207 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2023-07-08T13:57:24.443 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:24.448 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:24.448 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:24.449 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:25.423 INFO:teuthology.orchestra.run.smithi012.stdout:generic/333 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:57:25.461 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/333 2023-07-08T13:57:25.462 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/333 2023-07-08T13:57:25.462 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:25.465 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:25.488 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:25.489 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2023-07-08T13:57:25.723 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:25.727 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:25.728 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:25.729 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:25.729 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:26.666 INFO:teuthology.orchestra.run.smithi012.stdout:generic/334 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:57:26.729 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/334 2023-07-08T13:57:26.729 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/334 2023-07-08T13:57:26.729 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:26.731 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:26.759 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:26.760 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2023-07-08T13:57:26.995 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:26.999 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:27.000 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:27.001 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:27.001 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:27.356 INFO:teuthology.orchestra.run.smithi012.stdout:generic/335 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:57:27.360 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/335 2023-07-08T13:57:27.361 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/335 2023-07-08T13:57:27.361 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:27.363 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:27.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:27.395 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2023-07-08T13:57:27.619 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:27.624 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:27.624 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:27.625 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:27.625 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:28.105 INFO:teuthology.orchestra.run.smithi012.stdout:generic/336 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:57:28.109 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/336 2023-07-08T13:57:28.110 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/336 2023-07-08T13:57:28.110 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:28.112 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:28.136 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:28.137 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2023-07-08T13:57:28.357 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:28.361 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:28.362 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:28.363 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:28.363 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:29.356 INFO:teuthology.orchestra.run.smithi012.stdout:generic/337 1s 2023-07-08T13:57:29.459 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/337 2023-07-08T13:57:29.460 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:29.462 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:29.470 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:29.470 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2023-07-08T13:57:29.688 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:29.692 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:29.693 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:29.694 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:29.694 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:30.057 INFO:teuthology.orchestra.run.smithi012.stdout:generic/338 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:57:30.061 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/338 2023-07-08T13:57:30.062 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/338 2023-07-08T13:57:30.062 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:57:30.064 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:57:30.086 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:57:30.086 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2023-07-08T13:57:30.313 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:57:30.317 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:57:30.318 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:57:30.319 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:57:30.319 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:27.532 INFO:teuthology.orchestra.run.smithi012.stdout:generic/339 57s 2023-07-08T13:58:27.536 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/339 2023-07-08T13:58:27.536 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:27.547 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:27.547 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2023-07-08T13:58:27.799 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:27.803 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:27.804 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:27.805 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:27.805 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:31.880 INFO:teuthology.orchestra.run.smithi012.stdout:generic/340 3s 2023-07-08T13:58:31.947 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/340 2023-07-08T13:58:31.947 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:31.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:31.961 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2023-07-08T13:58:32.501 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:32.518 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:32.519 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:32.526 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:32.526 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:33.439 INFO:teuthology.orchestra.run.smithi012.stdout:generic/341 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:58:33.443 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/341 2023-07-08T13:58:33.443 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/341 2023-07-08T13:58:33.443 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:33.445 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:33.468 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:33.469 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2023-07-08T13:58:33.698 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:33.703 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:33.703 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:33.704 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:33.704 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:34.066 INFO:teuthology.orchestra.run.smithi012.stdout:generic/342 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:58:34.070 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/342 2023-07-08T13:58:34.070 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/342 2023-07-08T13:58:34.070 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:34.072 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:34.098 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:34.098 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2023-07-08T13:58:34.324 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:34.330 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:35.064 INFO:teuthology.orchestra.run.smithi012.stdout:generic/343 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:58:35.068 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/343 2023-07-08T13:58:35.068 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/343 2023-07-08T13:58:35.068 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:35.070 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:35.099 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:35.099 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2023-07-08T13:58:35.359 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:35.363 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:35.364 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:35.365 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:35.365 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:41.079 INFO:teuthology.orchestra.run.smithi012.stdout:generic/344 6s 2023-07-08T13:58:41.089 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/344 2023-07-08T13:58:41.090 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:41.115 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:41.164 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:41.164 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2023-07-08T13:58:41.700 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:41.708 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:41.709 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:41.710 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:47.702 INFO:teuthology.orchestra.run.smithi012.stdout:generic/345 5s 2023-07-08T13:58:47.720 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/345 2023-07-08T13:58:47.720 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:47.723 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:47.732 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:47.733 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2023-07-08T13:58:48.035 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:48.052 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:48.052 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:48.059 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:48.059 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:51.804 INFO:teuthology.orchestra.run.smithi012.stdout:generic/346 3s 2023-07-08T13:58:51.839 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/346 2023-07-08T13:58:51.839 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:51.841 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:51.850 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:51.850 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2023-07-08T13:58:52.264 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:52.269 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:52.270 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:52.271 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:52.271 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:52.823 INFO:teuthology.orchestra.run.smithi012.stdout:generic/347 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:58:52.827 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/347 2023-07-08T13:58:52.827 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/347 2023-07-08T13:58:52.828 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:52.829 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:52.858 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:52.859 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2023-07-08T13:58:53.082 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:53.086 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:53.086 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:53.087 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:53.088 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:53.439 INFO:teuthology.orchestra.run.smithi012.stdout:generic/348 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:58:53.443 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/348 2023-07-08T13:58:53.444 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/348 2023-07-08T13:58:53.444 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:53.446 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:53.472 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:53.473 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2023-07-08T13:58:53.703 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:53.708 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:53.708 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:53.709 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:53.709 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:54.199 INFO:teuthology.orchestra.run.smithi012.stdout:generic/349 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-07-08T13:58:54.203 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/349 2023-07-08T13:58:54.203 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/349 2023-07-08T13:58:54.203 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:54.205 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:54.237 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:54.238 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2023-07-08T13:58:54.458 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:54.463 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:54.463 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:54.464 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:54.464 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:55.420 INFO:teuthology.orchestra.run.smithi012.stdout:generic/350 1s 2023-07-08T13:58:55.425 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/350 2023-07-08T13:58:55.425 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:55.427 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:55.436 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:55.437 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2023-07-08T13:58:55.660 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:55.664 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:55.664 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:55.665 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:55.665 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:56.033 INFO:teuthology.orchestra.run.smithi012.stdout:generic/351 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T13:58:56.037 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/351 2023-07-08T13:58:56.038 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/351 2023-07-08T13:58:56.038 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:56.076 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:56.077 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2023-07-08T13:58:56.301 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:56.305 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:56.306 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:56.307 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:56.307 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:57.184 INFO:teuthology.orchestra.run.smithi012.stdout:generic/352 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:58:57.225 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/352 2023-07-08T13:58:57.226 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/352 2023-07-08T13:58:57.226 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:57.286 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:57.336 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:57.336 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2023-07-08T13:58:57.573 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:57.578 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:57.578 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:57.579 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:57.579 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:58.482 INFO:teuthology.orchestra.run.smithi012.stdout:generic/353 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:58:58.488 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/353 2023-07-08T13:58:58.489 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/353 2023-07-08T13:58:58.489 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:58:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:58:58.582 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:58:58.582 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2023-07-08T13:58:58.822 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:58:58.826 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:58:58.827 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:58:58.827 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:58:58.828 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:04.272 INFO:teuthology.orchestra.run.smithi012.stdout:generic/354 5s 2023-07-08T13:59:04.290 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/354 2023-07-08T13:59:04.291 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:04.302 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:04.314 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:04.315 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2023-07-08T13:59:04.852 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:04.857 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:04.858 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:04.861 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:04.861 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:05.925 INFO:teuthology.orchestra.run.smithi012.stdout:generic/355 0s 2023-07-08T13:59:05.930 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/355 2023-07-08T13:59:05.930 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:05.933 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:06.331 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:06.331 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/356) 2023-07-08T13:59:06.600 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:06.604 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:06.604 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:06.605 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:07.240 INFO:teuthology.orchestra.run.smithi012.stdout:generic/356 [not run] swapfiles are not supported 2023-07-08T13:59:07.244 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/356 2023-07-08T13:59:07.244 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/356 2023-07-08T13:59:07.245 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:07.246 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:07.269 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:07.270 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/357) 2023-07-08T13:59:07.496 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:07.500 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:07.500 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:07.501 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:07.502 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:08.092 INFO:teuthology.orchestra.run.smithi012.stdout:generic/357 [not run] swapfiles are not supported 2023-07-08T13:59:08.096 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/357 2023-07-08T13:59:08.097 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/357 2023-07-08T13:59:08.097 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:08.099 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:08.123 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:08.123 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/358) 2023-07-08T13:59:08.349 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:08.353 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:08.354 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:08.355 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:08.355 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:09.192 INFO:teuthology.orchestra.run.smithi012.stdout:generic/358 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:59:09.314 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/358 2023-07-08T13:59:09.314 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/358 2023-07-08T13:59:09.314 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:09.337 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:09.362 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:09.362 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/359) 2023-07-08T13:59:09.594 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:09.598 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:09.599 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:09.600 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:09.600 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:10.518 INFO:teuthology.orchestra.run.smithi012.stdout:generic/359 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:59:10.555 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/359 2023-07-08T13:59:10.556 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/359 2023-07-08T13:59:10.556 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:10.567 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:10.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:10.593 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/360) 2023-07-08T13:59:10.822 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:10.826 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:10.827 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:10.828 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:10.828 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:11.221 INFO:teuthology.orchestra.run.smithi012.stdout:generic/360 0s 2023-07-08T13:59:11.226 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/360 2023-07-08T13:59:11.226 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:11.228 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:11.255 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:11.256 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/361) 2023-07-08T13:59:11.482 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:11.486 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:11.486 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:11.487 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:11.488 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:11.850 INFO:teuthology.orchestra.run.smithi012.stdout:generic/361 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:59:11.854 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/361 2023-07-08T13:59:11.855 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/361 2023-07-08T13:59:11.855 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:11.856 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:11.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:11.881 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/362) 2023-07-08T13:59:12.113 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:12.117 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:12.118 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:12.119 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:12.119 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:12.455 INFO:teuthology.orchestra.run.smithi012.stdout:generic/362 [not run] this test requires richacl support on $SCRATCH_DEV 2023-07-08T13:59:12.459 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/362 2023-07-08T13:59:12.460 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/362 2023-07-08T13:59:12.460 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:12.462 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:12.489 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:12.489 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/363) 2023-07-08T13:59:12.716 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:12.720 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:12.721 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:12.722 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:12.722 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:13.058 INFO:teuthology.orchestra.run.smithi012.stdout:generic/363 [not run] this test requires richacl support on $SCRATCH_DEV 2023-07-08T13:59:13.062 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/363 2023-07-08T13:59:13.062 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/363 2023-07-08T13:59:13.062 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:13.064 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:13.088 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:13.089 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/364) 2023-07-08T13:59:13.312 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:13.317 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:13.317 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:13.318 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:13.665 INFO:teuthology.orchestra.run.smithi012.stdout:generic/364 [not run] this test requires richacl support on $SCRATCH_DEV 2023-07-08T13:59:13.669 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/364 2023-07-08T13:59:13.669 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/364 2023-07-08T13:59:13.669 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:13.671 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:13.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:13.693 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/365) 2023-07-08T13:59:13.936 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:13.940 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:13.941 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:13.942 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:13.942 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:generic/365 [not run] this test requires richacl support on $SCRATCH_DEV 2023-07-08T13:59:14.275 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/365 2023-07-08T13:59:14.275 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/365 2023-07-08T13:59:14.275 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:14.301 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:14.302 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/366) 2023-07-08T13:59:14.528 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:14.532 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:14.533 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:14.533 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:14.534 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:14.877 INFO:teuthology.orchestra.run.smithi012.stdout:generic/366 [not run] this test requires richacl support on $SCRATCH_DEV 2023-07-08T13:59:14.881 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/366 2023-07-08T13:59:14.881 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/366 2023-07-08T13:59:14.881 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:14.883 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:14.907 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:14.907 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/367) 2023-07-08T13:59:15.129 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:15.133 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:15.133 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:15.134 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:15.135 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:15.468 INFO:teuthology.orchestra.run.smithi012.stdout:generic/367 [not run] this test requires richacl support on $SCRATCH_DEV 2023-07-08T13:59:15.472 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/367 2023-07-08T13:59:15.472 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/367 2023-07-08T13:59:15.473 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:15.474 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:15.506 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:15.506 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/368) 2023-07-08T13:59:15.733 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:15.737 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:15.738 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:15.739 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:15.739 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:16.090 INFO:teuthology.orchestra.run.smithi012.stdout:generic/368 [not run] this test requires richacl support on $SCRATCH_DEV 2023-07-08T13:59:16.094 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/368 2023-07-08T13:59:16.095 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/368 2023-07-08T13:59:16.095 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:16.097 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:16.115 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:16.116 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/369) 2023-07-08T13:59:16.342 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:16.346 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:16.346 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:16.347 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:16.347 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:16.681 INFO:teuthology.orchestra.run.smithi012.stdout:generic/369 [not run] this test requires richacl support on $SCRATCH_DEV 2023-07-08T13:59:16.685 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/369 2023-07-08T13:59:16.685 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/369 2023-07-08T13:59:16.685 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:16.687 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:16.711 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:16.711 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/370) 2023-07-08T13:59:16.943 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:16.947 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:16.947 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:16.949 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:16.949 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:17.455 INFO:teuthology.orchestra.run.smithi012.stdout:generic/370 [not run] this test requires richacl support on $SCRATCH_DEV 2023-07-08T13:59:17.460 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/370 2023-07-08T13:59:17.460 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/370 2023-07-08T13:59:17.460 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:17.462 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:17.586 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:17.587 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/371) 2023-07-08T13:59:17.840 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:17.845 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:17.845 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:17.846 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:17.846 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:18.275 INFO:teuthology.orchestra.run.smithi012.stdout:generic/371 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T13:59:18.279 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/371 2023-07-08T13:59:18.280 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/371 2023-07-08T13:59:18.280 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:18.282 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:18.313 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:18.313 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/372) 2023-07-08T13:59:18.533 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:18.537 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:18.538 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:18.539 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:19.461 INFO:teuthology.orchestra.run.smithi012.stdout:generic/372 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:59:19.473 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/372 2023-07-08T13:59:19.473 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/372 2023-07-08T13:59:19.473 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:19.482 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:19.508 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:19.509 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/373) 2023-07-08T13:59:19.728 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:19.733 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:19.733 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:19.734 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:19.734 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:20.607 INFO:teuthology.orchestra.run.smithi012.stdout:generic/373 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:59:20.614 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/373 2023-07-08T13:59:20.614 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/373 2023-07-08T13:59:20.615 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:20.660 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:20.687 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:20.687 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/374) 2023-07-08T13:59:20.912 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:20.916 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:20.916 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:20.917 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:20.917 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:21.461 INFO:teuthology.orchestra.run.smithi012.stdout:generic/374 [not run] Dedupe not supported by scratch filesystem type: ceph 2023-07-08T13:59:21.772 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/374 2023-07-08T13:59:21.772 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/374 2023-07-08T13:59:21.772 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:21.839 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:21.897 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:21.898 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/375) 2023-07-08T13:59:22.128 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:22.133 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:22.133 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:22.134 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:22.134 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:22.555 INFO:teuthology.orchestra.run.smithi012.stdout:generic/375 0s 2023-07-08T13:59:22.560 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/375 2023-07-08T13:59:22.561 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:22.562 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:22.882 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:22.882 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/376) 2023-07-08T13:59:23.192 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:23.197 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:23.197 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:23.198 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:23.557 INFO:teuthology.orchestra.run.smithi012.stdout:generic/376 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:59:23.561 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/376 2023-07-08T13:59:23.561 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/376 2023-07-08T13:59:23.561 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:23.589 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:23.589 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/377) 2023-07-08T13:59:23.820 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:23.824 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:23.825 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:23.826 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:24.438 INFO:teuthology.orchestra.run.smithi012.stdout:generic/377 0s 2023-07-08T13:59:24.443 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/377 2023-07-08T13:59:24.443 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:24.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:24.454 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/378) 2023-07-08T13:59:24.678 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:24.683 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:24.683 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:24.684 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:25.163 INFO:teuthology.orchestra.run.smithi012.stdout:generic/378 1s 2023-07-08T13:59:25.168 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/378 2023-07-08T13:59:25.168 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:25.170 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:25.577 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:25.578 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/379) 2023-07-08T13:59:25.814 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:25.819 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:25.819 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:25.820 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:25.820 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:26.171 INFO:teuthology.orchestra.run.smithi012.stdout:generic/379 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:59:26.175 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/379 2023-07-08T13:59:26.175 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/379 2023-07-08T13:59:26.175 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:26.177 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:26.208 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:26.208 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/380) 2023-07-08T13:59:26.441 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:26.445 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:26.445 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:26.446 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:26.446 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:26.783 INFO:teuthology.orchestra.run.smithi012.stdout:generic/380 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:59:26.787 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/380 2023-07-08T13:59:26.787 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/380 2023-07-08T13:59:26.788 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:26.789 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:26.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:26.811 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/381) 2023-07-08T13:59:27.033 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:27.037 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:27.037 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:27.038 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:27.384 INFO:teuthology.orchestra.run.smithi012.stdout:generic/381 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:59:27.388 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/381 2023-07-08T13:59:27.388 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/381 2023-07-08T13:59:27.388 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:27.390 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:27.416 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:27.416 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/382) 2023-07-08T13:59:27.642 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:27.646 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:27.646 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:27.647 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:27.647 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:27.990 INFO:teuthology.orchestra.run.smithi012.stdout:generic/382 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:59:27.994 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/382 2023-07-08T13:59:27.995 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/382 2023-07-08T13:59:27.995 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:27.996 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:28.026 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:28.027 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/383) 2023-07-08T13:59:28.252 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:28.256 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:28.256 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:28.257 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:28.258 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:28.601 INFO:teuthology.orchestra.run.smithi012.stdout:generic/383 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:59:28.605 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/383 2023-07-08T13:59:28.605 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/383 2023-07-08T13:59:28.605 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:28.642 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:28.643 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/384) 2023-07-08T13:59:28.866 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:28.870 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:28.871 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:28.871 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:28.872 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:29.225 INFO:teuthology.orchestra.run.smithi012.stdout:generic/384 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:59:29.229 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/384 2023-07-08T13:59:29.230 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/384 2023-07-08T13:59:29.230 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:29.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:29.258 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/385) 2023-07-08T13:59:29.482 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:29.486 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:29.487 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:29.488 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:29.488 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:29.832 INFO:teuthology.orchestra.run.smithi012.stdout:generic/385 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:59:29.836 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/385 2023-07-08T13:59:29.836 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/385 2023-07-08T13:59:29.836 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:29.838 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:29.862 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:29.863 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/386) 2023-07-08T13:59:30.084 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:30.088 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:30.089 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:30.090 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:30.426 INFO:teuthology.orchestra.run.smithi012.stdout:generic/386 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:59:30.430 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/386 2023-07-08T13:59:30.430 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/386 2023-07-08T13:59:30.431 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:30.432 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:30.456 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:30.457 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/387) 2023-07-08T13:59:30.680 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:30.685 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:30.685 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:30.686 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:30.686 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:31.513 INFO:teuthology.orchestra.run.smithi012.stdout:generic/387 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:59:31.539 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/387 2023-07-08T13:59:31.539 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/387 2023-07-08T13:59:31.539 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:31.558 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:31.651 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:31.651 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/388) 2023-07-08T13:59:31.881 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:31.886 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:31.886 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:31.887 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:31.887 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:32.262 INFO:teuthology.orchestra.run.smithi012.stdout:generic/388 [not run] require 172.21.15.12:6789:/scratch to be local device 2023-07-08T13:59:32.266 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/388 2023-07-08T13:59:32.266 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/388 2023-07-08T13:59:32.266 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:32.297 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:32.297 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/389) 2023-07-08T13:59:32.526 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:32.530 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:32.531 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:32.532 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:32.532 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:32.879 INFO:teuthology.orchestra.run.smithi012.stdout:generic/389 [not run] O_TMPFILE is not supported 2023-07-08T13:59:32.883 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/389 2023-07-08T13:59:32.883 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/389 2023-07-08T13:59:32.883 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:32.885 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:32.907 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:32.908 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/390) 2023-07-08T13:59:33.132 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:33.137 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:33.137 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:33.138 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:33.138 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:33.501 INFO:teuthology.orchestra.run.smithi012.stdout:generic/390 [not run] ceph does not support freezing 2023-07-08T13:59:33.505 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/390 2023-07-08T13:59:33.505 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/390 2023-07-08T13:59:33.505 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:33.507 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:33.539 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:33.540 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/391) 2023-07-08T13:59:33.764 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:33.769 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:33.769 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:33.770 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:33.770 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:34.121 INFO:teuthology.orchestra.run.smithi012.stdout:generic/391 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T13:59:34.125 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/391 2023-07-08T13:59:34.125 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/391 2023-07-08T13:59:34.126 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:34.127 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:34.158 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:34.159 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/392) 2023-07-08T13:59:34.387 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:34.391 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:34.392 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:34.393 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:34.393 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:34.891 INFO:teuthology.orchestra.run.smithi012.stdout:generic/392 [not run] ceph does not support shutdown 2023-07-08T13:59:34.921 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/392 2023-07-08T13:59:34.921 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/392 2023-07-08T13:59:34.921 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:34.923 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:34.953 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:34.953 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/393) 2023-07-08T13:59:35.179 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:35.183 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:35.184 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:35.185 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:35.185 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:36.350 INFO:teuthology.orchestra.run.smithi012.stdout:generic/393 1s 2023-07-08T13:59:36.355 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/393 2023-07-08T13:59:36.355 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:36.357 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:36.366 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:36.366 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/394) 2023-07-08T13:59:36.595 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:36.599 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:36.599 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:36.600 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:36.600 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:36.987 INFO:teuthology.orchestra.run.smithi012.stdout:generic/394 0s 2023-07-08T13:59:36.992 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/394 2023-07-08T13:59:36.992 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:36.994 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:37.374 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:37.375 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/395) 2023-07-08T13:59:37.629 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:37.633 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:37.634 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:37.635 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:37.635 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:generic/395 [not run] kernel does not support ceph encryption 2023-07-08T13:59:38.419 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/395 2023-07-08T13:59:38.419 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/395 2023-07-08T13:59:38.419 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:38.422 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:38.466 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:38.467 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/396) 2023-07-08T13:59:38.852 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:38.857 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:38.857 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:38.858 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:38.858 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:39.691 INFO:teuthology.orchestra.run.smithi012.stdout:generic/396 [not run] kernel does not support ceph encryption 2023-07-08T13:59:39.713 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/396 2023-07-08T13:59:39.713 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/396 2023-07-08T13:59:39.713 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:39.715 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:39.735 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:39.736 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/397) 2023-07-08T13:59:39.960 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:39.964 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:39.965 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:39.966 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:40.539 INFO:teuthology.orchestra.run.smithi012.stdout:generic/397 [not run] kernel does not support ceph encryption 2023-07-08T13:59:40.557 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/397 2023-07-08T13:59:40.557 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/397 2023-07-08T13:59:40.558 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:40.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:40.593 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/398) 2023-07-08T13:59:40.816 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:40.820 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:40.821 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:40.822 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:41.396 INFO:teuthology.orchestra.run.smithi012.stdout:generic/398 [not run] kernel does not support ceph encryption 2023-07-08T13:59:41.418 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/398 2023-07-08T13:59:41.418 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/398 2023-07-08T13:59:41.418 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:41.420 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:41.446 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:41.447 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/399) 2023-07-08T13:59:41.672 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:41.676 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:41.677 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:41.677 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:41.678 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:42.263 INFO:teuthology.orchestra.run.smithi012.stdout:generic/399 [not run] kernel does not support ceph encryption 2023-07-08T13:59:42.287 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/399 2023-07-08T13:59:42.287 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/399 2023-07-08T13:59:42.287 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:42.312 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:42.313 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/400) 2023-07-08T13:59:42.534 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:42.539 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:42.539 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:42.540 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:42.883 INFO:teuthology.orchestra.run.smithi012.stdout:generic/400 [not run] disk quotas not supported by this filesystem type: ceph 2023-07-08T13:59:42.887 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/400 2023-07-08T13:59:42.887 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/400 2023-07-08T13:59:42.888 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:42.889 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:42.918 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:42.919 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/401) 2023-07-08T13:59:43.144 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:43.148 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:43.148 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:43.149 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:44.200 INFO:teuthology.orchestra.run.smithi012.stdout:generic/401 1s 2023-07-08T13:59:44.221 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/401 2023-07-08T13:59:44.222 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:44.224 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:44.232 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:44.233 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/402) 2023-07-08T13:59:44.458 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:44.463 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:44.463 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:44.464 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:44.464 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:45.025 INFO:teuthology.orchestra.run.smithi012.stdout:generic/402 [not run] filesystem ceph timestamp bounds are unknown 2023-07-08T13:59:45.042 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/402 2023-07-08T13:59:45.042 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/402 2023-07-08T13:59:45.043 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:45.081 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:45.082 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/403) 2023-07-08T13:59:45.305 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:45.309 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:45.309 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:45.310 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:46.057 INFO:teuthology.orchestra.run.smithi012.stdout:generic/403 1s 2023-07-08T13:59:46.062 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/403 2023-07-08T13:59:46.062 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:46.064 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:46.073 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:46.073 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/404) 2023-07-08T13:59:46.295 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:46.299 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:46.299 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:46.300 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:46.301 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:46.658 INFO:teuthology.orchestra.run.smithi012.stdout:generic/404 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T13:59:46.662 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/404 2023-07-08T13:59:46.663 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/404 2023-07-08T13:59:46.663 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:46.697 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:46.698 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/405) 2023-07-08T13:59:46.920 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:46.925 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:46.925 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:46.926 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:47.314 INFO:teuthology.orchestra.run.smithi012.stdout:generic/405 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T13:59:47.318 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/405 2023-07-08T13:59:47.319 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/405 2023-07-08T13:59:47.319 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:47.321 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:47.344 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:47.344 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/406) 2023-07-08T13:59:47.570 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:47.574 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:47.574 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:47.575 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:47.575 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:48.136 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:59:48.139 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T13:59:48.178 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-07-08T13:59:48.177+0000 7f83fa332700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 439579) UID: 0 2023-07-08T13:59:48.179 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-07-08T13:59:48.177+0000 7fe6c1b3c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 439579) UID: 0 2023-07-08T13:59:48.179 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-07-08T13:59:48.177+0000 7f589b052700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 439579) UID: 0 2023-07-08T13:59:48.179 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-07-08T13:59:48.177+0000 7fcf305dc700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 439579) UID: 0 2023-07-08T13:59:49.459 INFO:teuthology.orchestra.run.smithi012.stdout:generic/406 2s 2023-07-08T13:59:49.464 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/406 2023-07-08T13:59:49.464 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:49.475 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:49.475 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/407) 2023-07-08T13:59:49.745 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:49.749 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:49.749 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:49.750 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:49.750 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:50.219 INFO:teuthology.orchestra.run.smithi012.stdout:generic/407 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T13:59:50.223 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/407 2023-07-08T13:59:50.223 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/407 2023-07-08T13:59:50.224 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:50.226 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:50.556 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:50.556 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/408) 2023-07-08T13:59:50.893 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:50.897 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:50.898 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:50.899 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:51.300 INFO:teuthology.orchestra.run.smithi012.stdout:generic/408 [not run] Dedupe not supported by test filesystem type: ceph 2023-07-08T13:59:51.305 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/408 2023-07-08T13:59:51.305 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/408 2023-07-08T13:59:51.305 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:51.610 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:51.611 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/409) 2023-07-08T13:59:51.965 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:51.969 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:51.970 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:51.971 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:51.971 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:52.318 INFO:teuthology.orchestra.run.smithi012.stdout:generic/409 [not run] require 172.21.15.12:6789:/scratch to be local device 2023-07-08T13:59:52.322 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/409 2023-07-08T13:59:52.322 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/409 2023-07-08T13:59:52.323 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:52.325 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:52.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:52.349 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/410) 2023-07-08T13:59:52.576 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:52.580 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:52.581 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:52.582 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:52.582 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:52.934 INFO:teuthology.orchestra.run.smithi012.stdout:generic/410 [not run] require 172.21.15.12:6789:/scratch to be local device 2023-07-08T13:59:52.938 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/410 2023-07-08T13:59:52.938 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/410 2023-07-08T13:59:52.938 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:52.940 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:52.964 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:52.965 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/411) 2023-07-08T13:59:53.195 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:53.200 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:53.200 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:53.204 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:53.204 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:53.565 INFO:teuthology.orchestra.run.smithi012.stdout:generic/411 [not run] require 172.21.15.12:6789:/scratch to be local device 2023-07-08T13:59:53.569 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/411 2023-07-08T13:59:53.569 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/411 2023-07-08T13:59:53.569 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:53.571 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:53.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:53.596 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/412) 2023-07-08T13:59:53.822 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:53.826 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:53.827 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:53.828 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:53.828 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:54.627 INFO:teuthology.orchestra.run.smithi012.stdout:generic/412 0s 2023-07-08T13:59:54.632 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/412 2023-07-08T13:59:54.633 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:54.634 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:54.643 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:54.643 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/413) 2023-07-08T13:59:54.869 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:54.873 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:54.874 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:54.875 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:54.875 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:55.372 INFO:teuthology.orchestra.run.smithi012.stdout:generic/413 [not run] mount 172.21.15.12:6789:/scratch with dax failed 2023-07-08T13:59:55.376 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/413 2023-07-08T13:59:55.376 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/413 2023-07-08T13:59:55.376 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:55.378 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:55.403 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:55.404 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/414) 2023-07-08T13:59:55.648 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:55.652 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:55.653 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:55.654 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:55.654 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:generic/414 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:59:56.612 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/414 2023-07-08T13:59:56.613 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/414 2023-07-08T13:59:56.613 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:56.617 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:56.650 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:56.650 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/415) 2023-07-08T13:59:56.876 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:56.881 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:56.881 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:56.882 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:56.882 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:57.822 INFO:teuthology.orchestra.run.smithi012.stdout:generic/415 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T13:59:57.828 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/415 2023-07-08T13:59:57.829 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/415 2023-07-08T13:59:57.829 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:57.832 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:57.865 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:57.866 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/416) 2023-07-08T13:59:58.099 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:58.103 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:58.103 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:58.105 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:58.105 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:58.446 INFO:teuthology.orchestra.run.smithi012.stdout:generic/416 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T13:59:58.450 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/416 2023-07-08T13:59:58.450 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/416 2023-07-08T13:59:58.450 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:58.452 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:58.479 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:58.479 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/417) 2023-07-08T13:59:58.703 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:58.707 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:58.707 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:58.708 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:58.709 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:59.228 INFO:teuthology.orchestra.run.smithi012.stdout:generic/417 [not run] ceph does not support shutdown 2023-07-08T13:59:59.254 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/417 2023-07-08T13:59:59.254 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/417 2023-07-08T13:59:59.254 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:59.255 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:59.279 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:59.280 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/418) 2023-07-08T13:59:59.515 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T13:59:59.519 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T13:59:59.519 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T13:59:59.521 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T13:59:59.521 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:59.935 INFO:teuthology.orchestra.run.smithi012.stdout:generic/418 [not run] require 172.21.15.12:6789:/test to be valid block disk 2023-07-08T13:59:59.939 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/418 2023-07-08T13:59:59.939 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/418 2023-07-08T13:59:59.940 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T13:59:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T13:59:59.979 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T13:59:59.980 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/419) 2023-07-08T14:00:00.217 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:00:00.221 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:00:00.222 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:00:00.223 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:00:00.223 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:00.803 INFO:teuthology.orchestra.run.smithi012.stdout:generic/419 [not run] kernel does not support ceph encryption 2023-07-08T14:00:00.822 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/419 2023-07-08T14:00:00.822 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/419 2023-07-08T14:00:00.822 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:00:00.824 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:00.857 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:00:00.857 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/420) 2023-07-08T14:00:01.086 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:00:01.091 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:00:01.091 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:00:01.092 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:00:01.092 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:01.583 INFO:teuthology.orchestra.run.smithi012.stdout:generic/420 0s 2023-07-08T14:00:01.605 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/420 2023-07-08T14:00:01.605 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:00:01.607 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:02.033 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:00:02.033 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/421) 2023-07-08T14:00:02.274 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:00:02.278 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:00:02.279 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:00:02.280 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:00:02.280 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:02.851 INFO:teuthology.orchestra.run.smithi012.stdout:generic/421 [not run] kernel does not support ceph encryption 2023-07-08T14:00:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/421 2023-07-08T14:00:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/421 2023-07-08T14:00:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:00:02.875 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:02.900 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:00:02.901 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/422) 2023-07-08T14:00:03.127 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:00:03.131 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:00:03.131 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:00:03.132 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:00:03.133 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:03.509 INFO:teuthology.orchestra.run.smithi012.stdout:generic/422 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-07-08T14:00:03.513 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/422 2023-07-08T14:00:03.513 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/422 2023-07-08T14:00:03.514 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:00:03.515 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:03.547 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:00:03.548 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/423) 2023-07-08T14:00:03.775 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:00:03.779 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:00:03.780 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:00:03.781 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:00:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:04.485 INFO:teuthology.orchestra.run.smithi012.stdout:generic/423 1s 2023-07-08T14:00:04.492 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/423 2023-07-08T14:00:04.493 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:00:04.494 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:04.874 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:00:04.874 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/424) 2023-07-08T14:00:05.191 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:00:05.195 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:00:05.196 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:00:05.197 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:00:05.197 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:05.565 INFO:teuthology.orchestra.run.smithi012.stdout:generic/424 [not run] file system doesn't support any of /bin/chattr +a/+c/+d/+i 2023-07-08T14:00:05.569 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/424 2023-07-08T14:00:05.569 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/424 2023-07-08T14:00:05.569 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:00:05.571 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:05.603 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:00:05.604 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/425) 2023-07-08T14:00:05.829 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:00:05.833 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:00:05.834 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:00:05.835 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:00:05.835 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:06.217 INFO:teuthology.orchestra.run.smithi012.stdout:generic/425 [not run] xfs_io fiemap -a failed (old kernel/wrong fs?) 2023-07-08T14:00:06.221 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/425 2023-07-08T14:00:06.221 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/425 2023-07-08T14:00:06.221 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:00:06.223 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:00:06.526 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:00:06.526 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/426) 2023-07-08T14:00:06.897 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:00:06.902 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:00:06.902 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:00:06.903 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:00:06.903 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:03.405 INFO:teuthology.orchestra.run.smithi012.stdout:generic/426 56s 2023-07-08T14:01:03.566 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/426 2023-07-08T14:01:03.567 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:01:03.569 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:04.055 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:01:04.056 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/427) 2023-07-08T14:01:06.327 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:01:06.331 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:01:06.332 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:01:06.332 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:01:06.332 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:06.857 INFO:teuthology.orchestra.run.smithi012.stdout:generic/427 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T14:01:06.861 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/427 2023-07-08T14:01:06.862 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/427 2023-07-08T14:01:06.862 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:01:06.864 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:06.892 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:01:06.893 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/428) 2023-07-08T14:01:07.131 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:01:07.135 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:01:07.135 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:01:07.136 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:01:07.137 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:07.582 INFO:teuthology.orchestra.run.smithi012.stdout:generic/428 0s 2023-07-08T14:01:07.587 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/428 2023-07-08T14:01:07.587 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:01:07.589 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:07.627 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:01:07.627 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/429) 2023-07-08T14:01:07.877 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:01:07.881 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:01:07.881 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:01:07.882 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:01:07.882 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:08.461 INFO:teuthology.orchestra.run.smithi012.stdout:generic/429 [not run] kernel does not support ceph encryption 2023-07-08T14:01:08.484 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/429 2023-07-08T14:01:08.484 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/429 2023-07-08T14:01:08.484 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:01:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:08.510 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:01:08.511 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/430) 2023-07-08T14:01:08.744 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:01:08.749 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:01:08.749 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:01:08.750 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:01:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:09.247 INFO:teuthology.orchestra.run.smithi012.stdout:generic/430 1s 2023-07-08T14:01:09.252 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/430 2023-07-08T14:01:09.252 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:01:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:09.500 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:01:09.501 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/431) 2023-07-08T14:01:09.962 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:01:09.966 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:01:09.967 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:01:09.968 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:01:09.968 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:10.404 INFO:teuthology.orchestra.run.smithi012.stdout:generic/431 0s 2023-07-08T14:01:10.409 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/431 2023-07-08T14:01:10.409 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:01:10.411 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:10.739 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:01:10.740 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/432) 2023-07-08T14:01:11.102 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:01:11.106 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:01:11.107 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:01:11.108 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:01:11.108 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:11.580 INFO:teuthology.orchestra.run.smithi012.stdout:generic/432 0s 2023-07-08T14:01:11.585 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/432 2023-07-08T14:01:11.585 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:01:11.587 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:11.996 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:01:11.996 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/433) 2023-07-08T14:01:12.271 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:01:12.276 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:01:12.276 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:01:12.277 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:01:12.277 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:12.692 INFO:teuthology.orchestra.run.smithi012.stdout:generic/433 0s 2023-07-08T14:01:12.698 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/433 2023-07-08T14:01:12.698 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:01:12.700 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:13.048 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:01:13.049 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/434) 2023-07-08T14:01:13.414 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:01:13.418 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:01:13.418 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:01:13.419 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:01:13.419 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:13.932 INFO:teuthology.orchestra.run.smithi012.stdout:generic/434 0s 2023-07-08T14:01:13.936 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/434 2023-07-08T14:01:13.937 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:01:13.939 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:14.060 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:01:14.060 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/435) 2023-07-08T14:01:14.632 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:01:14.636 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:01:14.636 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:01:14.637 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:01:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:15.207 INFO:teuthology.orchestra.run.smithi012.stdout:generic/435 [not run] kernel does not support ceph encryption 2023-07-08T14:01:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/435 2023-07-08T14:01:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/435 2023-07-08T14:01:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:01:15.238 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:15.264 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:01:15.264 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/436) 2023-07-08T14:01:15.491 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:01:15.495 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:01:15.496 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:01:15.497 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:01:15.497 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:16.269 INFO:teuthology.orchestra.run.smithi012.stdout:generic/436 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T14:01:16.273 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/436 2023-07-08T14:01:16.274 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/436 2023-07-08T14:01:16.274 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:01:16.275 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:16.606 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:01:16.606 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/437) 2023-07-08T14:01:16.926 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:01:16.930 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:01:16.931 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:01:16.932 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:01:16.932 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:17.936 INFO:teuthology.orchestra.run.smithi012.stdout:generic/437 0s 2023-07-08T14:01:17.941 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/437 2023-07-08T14:01:17.942 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:01:17.943 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:01:17.974 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:01:17.975 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/438) 2023-07-08T14:01:18.204 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:01:18.208 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:01:18.208 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:01:18.209 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:01:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:34.503 INFO:teuthology.orchestra.run.smithi012.stdout:generic/438 76s 2023-07-08T14:02:34.508 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/438 2023-07-08T14:02:34.508 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:02:34.510 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:34.545 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:02:34.545 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/439) 2023-07-08T14:02:34.771 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:02:34.775 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:02:34.776 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:02:34.777 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:02:34.777 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:35.957 INFO:teuthology.orchestra.run.smithi012.stdout:generic/439 1s 2023-07-08T14:02:35.962 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/439 2023-07-08T14:02:35.963 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:02:35.965 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:35.990 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:02:35.991 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/440) 2023-07-08T14:02:36.228 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:02:36.233 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:02:36.233 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:02:36.234 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:02:36.234 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:36.789 INFO:teuthology.orchestra.run.smithi012.stdout:generic/440 [not run] kernel does not support ceph encryption 2023-07-08T14:02:36.806 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/440 2023-07-08T14:02:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/440 2023-07-08T14:02:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:02:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:36.830 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:02:36.830 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/441) 2023-07-08T14:02:37.055 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:02:37.059 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:02:37.059 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:02:37.060 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:02:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:37.495 INFO:teuthology.orchestra.run.smithi012.stdout:generic/441 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T14:02:37.499 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/441 2023-07-08T14:02:37.499 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/441 2023-07-08T14:02:37.499 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:02:37.501 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:37.525 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:02:37.526 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/442) 2023-07-08T14:02:37.749 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:02:37.753 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:02:37.754 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:02:37.755 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:02:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:38.114 INFO:teuthology.orchestra.run.smithi012.stdout:generic/442 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T14:02:38.118 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/442 2023-07-08T14:02:38.118 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/442 2023-07-08T14:02:38.118 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:02:38.120 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:38.138 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:02:38.138 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/443) 2023-07-08T14:02:38.366 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:02:38.370 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:02:38.371 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:02:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:02:38.372 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:38.761 INFO:teuthology.orchestra.run.smithi012.stdout:generic/443 0s 2023-07-08T14:02:38.766 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/443 2023-07-08T14:02:38.767 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:02:38.769 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:38.795 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:02:38.796 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/444) 2023-07-08T14:02:39.020 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:02:39.024 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:02:39.025 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:02:39.026 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:02:39.026 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:39.456 INFO:teuthology.orchestra.run.smithi012.stdout:generic/444 0s 2023-07-08T14:02:39.463 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/444 2023-07-08T14:02:39.463 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:02:39.465 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:39.508 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:02:39.508 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/445) 2023-07-08T14:02:39.734 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:02:39.738 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:02:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:02:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:02:39.740 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:40.442 INFO:teuthology.orchestra.run.smithi012.stdout:generic/445 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T14:02:40.447 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/445 2023-07-08T14:02:40.447 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/445 2023-07-08T14:02:40.447 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:02:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:02:40.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:02:40.841 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/446) 2023-07-08T14:02:41.076 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:02:41.081 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:02:41.081 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:02:41.082 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:02:41.082 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:03.881 INFO:teuthology.orchestra.run.smithi012.stdout:generic/446 22s 2023-07-08T14:03:03.886 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/446 2023-07-08T14:03:03.887 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:03.892 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:03.901 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:03.902 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/447) 2023-07-08T14:03:03.919 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:03:03.946 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:03:04.016 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-07-08T14:03:04.015+0000 7f83fa332700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 485503) UID: 0 2023-07-08T14:03:04.017 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-07-08T14:03:04.015+0000 7fcf305dc700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 485503) UID: 0 2023-07-08T14:03:04.017 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-07-08T14:03:04.015+0000 7fe6c1b3c700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 485503) UID: 0 2023-07-08T14:03:04.017 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-07-08T14:03:04.015+0000 7f589b052700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 485503) UID: 0 2023-07-08T14:03:04.134 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:04.139 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:04.139 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:04.140 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:04.140 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:04.905 INFO:teuthology.orchestra.run.smithi012.stdout:generic/447 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T14:03:04.923 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/447 2023-07-08T14:03:04.923 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/447 2023-07-08T14:03:04.923 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:05.013 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:05.124 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:05.125 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/448) 2023-07-08T14:03:05.381 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:05.385 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:05.386 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:05.387 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:05.387 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:06.362 INFO:teuthology.orchestra.run.smithi012.stdout:generic/448 1s 2023-07-08T14:03:06.367 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/448 2023-07-08T14:03:06.367 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:06.369 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:06.787 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:06.787 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/449) 2023-07-08T14:03:07.024 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:07.028 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:07.028 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:07.029 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:07.029 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:07.424 INFO:teuthology.orchestra.run.smithi012.stdout:generic/449 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2023-07-08T14:03:07.428 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/449 2023-07-08T14:03:07.428 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/449 2023-07-08T14:03:07.429 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:07.430 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:07.880 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:07.880 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/450) 2023-07-08T14:03:08.114 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:08.118 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:08.118 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:08.119 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:08.119 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:08.885 INFO:teuthology.orchestra.run.smithi012.stdout:generic/450 0s 2023-07-08T14:03:08.889 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/450 2023-07-08T14:03:08.890 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:08.892 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:09.263 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:09.264 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/451) 2023-07-08T14:03:09.547 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:09.552 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:09.552 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:09.553 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:09.553 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:40.147 INFO:teuthology.orchestra.run.smithi012.stdout:generic/451 31s 2023-07-08T14:03:40.202 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/451 2023-07-08T14:03:40.202 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:40.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:40.565 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/452) 2023-07-08T14:03:40.874 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:40.879 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:40.879 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:40.880 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:41.491 INFO:teuthology.orchestra.run.smithi012.stdout:generic/452 0s 2023-07-08T14:03:41.496 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/452 2023-07-08T14:03:41.496 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:41.498 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:41.506 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:41.507 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/453) 2023-07-08T14:03:41.728 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:41.732 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:41.732 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:41.733 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:42.065 INFO:teuthology.orchestra.run.smithi012.stdout:generic/453 [not run] ceph does not allow unrestricted byte streams for names 2023-07-08T14:03:42.069 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/453 2023-07-08T14:03:42.069 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/453 2023-07-08T14:03:42.070 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:42.071 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:42.092 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:42.093 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/454) 2023-07-08T14:03:42.322 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:42.326 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:42.327 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:42.328 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:42.673 INFO:teuthology.orchestra.run.smithi012.stdout:generic/454 [not run] ceph does not allow unrestricted byte streams for names 2023-07-08T14:03:42.677 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/454 2023-07-08T14:03:42.677 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/454 2023-07-08T14:03:42.678 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:42.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:42.712 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/455) 2023-07-08T14:03:42.935 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:42.940 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:42.940 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:42.941 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:43.367 INFO:teuthology.orchestra.run.smithi012.stdout:generic/455 [not run] This test requires a valid $LOGWRITES_DEV 2023-07-08T14:03:43.371 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/455 2023-07-08T14:03:43.371 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/455 2023-07-08T14:03:43.371 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:43.398 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:43.398 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/456) 2023-07-08T14:03:43.624 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:43.628 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:43.629 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:43.630 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:43.997 INFO:teuthology.orchestra.run.smithi012.stdout:generic/456 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-07-08T14:03:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/456 2023-07-08T14:03:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/456 2023-07-08T14:03:44.002 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:44.003 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:44.032 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:44.032 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/457) 2023-07-08T14:03:44.264 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:44.268 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:44.268 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:44.269 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:44.270 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:45.275 INFO:teuthology.orchestra.run.smithi012.stdout:generic/457 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T14:03:45.282 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/457 2023-07-08T14:03:45.282 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/457 2023-07-08T14:03:45.282 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:45.310 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:45.310 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/458) 2023-07-08T14:03:45.542 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:45.546 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:45.546 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:45.547 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:45.548 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:46.457 INFO:teuthology.orchestra.run.smithi012.stdout:generic/458 [not run] Reflink not supported by scratch filesystem type: ceph 2023-07-08T14:03:46.464 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/458 2023-07-08T14:03:46.465 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/458 2023-07-08T14:03:46.465 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:46.507 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:46.536 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:46.536 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/459) 2023-07-08T14:03:46.764 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:46.768 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:46.768 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:46.769 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:46.769 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:47.552 INFO:teuthology.orchestra.run.smithi012.stdout:generic/459 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T14:03:47.556 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/459 2023-07-08T14:03:47.556 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/459 2023-07-08T14:03:47.556 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:47.558 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:47.588 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:47.589 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/460) 2023-07-08T14:03:47.821 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:47.826 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:47.826 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:47.827 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:47.827 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:49.969 INFO:teuthology.orchestra.run.smithi012.stdout:generic/460 1s 2023-07-08T14:03:49.974 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/460 2023-07-08T14:03:49.975 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:49.985 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:49.985 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/461) 2023-07-08T14:03:50.211 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:50.215 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:50.216 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:50.217 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:50.217 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:50.743 INFO:teuthology.orchestra.run.smithi012.stdout:generic/461 [not run] ceph does not support shutdown 2023-07-08T14:03:50.747 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/461 2023-07-08T14:03:50.747 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/461 2023-07-08T14:03:50.747 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:50.749 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:50.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:50.773 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/462) 2023-07-08T14:03:50.998 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:51.003 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:51.003 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:51.004 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:51.004 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:51.686 INFO:teuthology.orchestra.run.smithi012.stdout:generic/462 [not run] mount 172.21.15.12:6789:/scratch with dax failed 2023-07-08T14:03:51.700 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/462 2023-07-08T14:03:51.700 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/462 2023-07-08T14:03:51.700 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:51.748 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:51.749 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/463) 2023-07-08T14:03:51.994 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:51.998 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:51.998 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:51.999 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:52.000 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:52.423 INFO:teuthology.orchestra.run.smithi012.stdout:generic/463 [not run] Reflink not supported by test filesystem type: ceph 2023-07-08T14:03:52.427 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/463 2023-07-08T14:03:52.428 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/463 2023-07-08T14:03:52.428 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:03:52.430 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:03:52.853 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:03:52.854 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/464) 2023-07-08T14:03:53.089 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:03:53.093 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:03:53.093 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:03:53.094 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:03:53.095 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:04:51.007 INFO:teuthology.orchestra.run.smithi012.stdout:generic/464 57s 2023-07-08T14:04:51.012 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/464 2023-07-08T14:04:51.012 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:04:51.015 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:04:51.024 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:04:51.025 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/465) 2023-07-08T14:04:51.254 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:04:51.258 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:04:51.258 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:04:51.259 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:04:51.259 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:04:56.226 INFO:teuthology.orchestra.run.smithi012.stdout:generic/465 5s 2023-07-08T14:04:56.235 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/465 2023-07-08T14:04:56.236 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:04:56.238 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:04:56.250 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:04:56.251 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/466) 2023-07-08T14:04:56.487 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:04:56.491 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:04:56.492 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:04:56.492 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:04:56.493 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:04:56.925 INFO:teuthology.orchestra.run.smithi012.stdout:generic/466 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T14:04:56.929 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/466 2023-07-08T14:04:56.929 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/466 2023-07-08T14:04:56.929 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:04:56.931 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:04:56.955 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:04:56.955 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/467) 2023-07-08T14:04:57.197 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:04:57.201 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:04:57.202 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:04:57.203 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:04:57.203 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:02.691 INFO:teuthology.orchestra.run.smithi012.stdout:generic/467 65s 2023-07-08T14:06:02.775 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/467 2023-07-08T14:06:02.776 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:06:02.786 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:03.271 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:06:03.271 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/468) 2023-07-08T14:06:04.228 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:06:04.232 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:06:04.233 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:06:04.234 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:06:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:04.906 INFO:teuthology.orchestra.run.smithi012.stdout:generic/468 [not run] ceph does not support shutdown 2023-07-08T14:06:04.924 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/468 2023-07-08T14:06:04.924 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/468 2023-07-08T14:06:04.924 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:06:04.926 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:04.948 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:06:04.948 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/469) 2023-07-08T14:06:05.184 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:06:05.188 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:06:05.188 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:06:05.189 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:06:05.190 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:05.545 INFO:teuthology.orchestra.run.smithi012.stdout:generic/469 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2023-07-08T14:06:05.549 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/469 2023-07-08T14:06:05.549 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/469 2023-07-08T14:06:05.549 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:06:05.551 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:05.576 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:06:05.576 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/470) 2023-07-08T14:06:05.811 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:06:05.815 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:06:05.816 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:06:05.817 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:06:05.817 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:06.203 INFO:teuthology.orchestra.run.smithi012.stdout:generic/470 [not run] This test requires a valid $LOGWRITES_DEV 2023-07-08T14:06:06.207 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/470 2023-07-08T14:06:06.207 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/470 2023-07-08T14:06:06.208 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:06:06.209 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:06.237 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:06:06.237 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/471) 2023-07-08T14:06:06.471 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:06:06.475 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:06:06.476 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:06:06.477 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:06:06.477 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:06.890 INFO:teuthology.orchestra.run.smithi012.stdout:generic/471 [not run] xfs_io pwrite -V 1 -b 4k -N failed (old kernel/wrong fs?) 2023-07-08T14:06:06.894 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/471 2023-07-08T14:06:06.894 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/471 2023-07-08T14:06:06.894 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:06:06.896 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:07.291 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:06:07.292 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/472) 2023-07-08T14:06:07.556 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:06:07.560 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:06:07.561 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:06:07.562 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:06:07.562 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:08.199 INFO:teuthology.orchestra.run.smithi012.stdout:generic/472 [not run] swapfiles are not supported 2023-07-08T14:06:08.203 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/472 2023-07-08T14:06:08.203 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/472 2023-07-08T14:06:08.203 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:06:08.205 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:08.235 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:06:08.236 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/473) 2023-07-08T14:06:08.461 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:06:08.465 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:06:08.466 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:06:08.467 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:06:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:08.852 INFO:teuthology.orchestra.run.smithi012.stdout:generic/473 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-07-08T14:06:08.856 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/473 2023-07-08T14:06:08.856 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/473 2023-07-08T14:06:08.856 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:06:08.858 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:08.888 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:06:08.888 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/474) 2023-07-08T14:06:09.113 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:06:09.117 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:06:09.118 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:06:09.119 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:06:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:09.631 INFO:teuthology.orchestra.run.smithi012.stdout:generic/474 [not run] ceph does not support shutdown 2023-07-08T14:06:09.650 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/474 2023-07-08T14:06:09.651 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/474 2023-07-08T14:06:09.651 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:06:09.653 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:09.681 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:06:09.682 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/475) 2023-07-08T14:06:09.906 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:06:09.911 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:06:09.911 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:06:09.912 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:06:09.912 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:10.291 INFO:teuthology.orchestra.run.smithi012.stdout:generic/475 [not run] require 172.21.15.12:6789:/scratch to be valid block disk 2023-07-08T14:06:10.295 INFO:teuthology.orchestra.run.smithi012.stdout:Ran: generic/475 2023-07-08T14:06:10.296 INFO:teuthology.orchestra.run.smithi012.stdout:Not run: generic/475 2023-07-08T14:06:10.296 INFO:teuthology.orchestra.run.smithi012.stdout:Passed all 1 tests 2023-07-08T14:06:10.298 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:10.322 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-07-08T14:06:10.323 DEBUG:teuthology.orchestra.run.smithi012:> (cd /tmp/tmp.HvAMMOKiw7xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/476) 2023-07-08T14:06:10.552 INFO:teuthology.orchestra.run.smithi012.stdout:FSTYP -- ceph 2023-07-08T14:06:10.556 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM -- Linux/x86_64 smithi012 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-07-08T14:06:10.557 INFO:teuthology.orchestra.run.smithi012.stdout:MKFS_OPTIONS -- 172.21.15.12:6789:/scratch 2023-07-08T14:06:10.558 INFO:teuthology.orchestra.run.smithi012.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQB0WKlkXt9GAxAAqt8jqiT+6X0+Am8ZIo7M0g== -o context=system_u:object_r:root_t:s0 172.21.15.12:6789:/scratch /tmp/tmp.0MVtsqxQ3pscratch 2023-07-08T14:06:10.558 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-07-08T14:06:24.457 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:06:24.459 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:06:54.760 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:06:54.786 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:07:25.442 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:07:25.461 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:07:56.005 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:07:56.007 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:08:26.082 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:08:26.128 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:08:56.537 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:08:56.601 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:09:27.050 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:09:27.149 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:09:57.371 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:09:57.374 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:10:27.547 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:10:27.550 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:10:57.874 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:10:57.895 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:11:28.705 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:11:28.741 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:11:58.995 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:11:59.104 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-08T14:12:06.907 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:12:06.907 INFO:teuthology.orchestra.run.smithi012.stdout:generic/476 2023-07-08T14:12:06.908 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-07-08T14:12:06.908 INFO:tasks.cephfs.xfstests_dev:Command return value: -1 2023-07-08T14:12:06.909 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:12:06.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:12:06.912 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:12:08.652 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-07-08T14:12:08.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-07-08T14:12:08.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-07-08T14:12:08.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-07-08T14:12:08.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2023-07-08T14:12:08.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-07-08T14:12:08.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-07-08T14:12:08.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-07-08T14:12:08.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-07-08T14:12:08.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-07-08T14:12:08.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-07-08T14:12:08.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-07-08T14:12:08.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-07-08T14:12:08.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-07-08T14:12:08.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-07-08T14:12:08.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2023-07-08T14:12:08.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2023-07-08T14:12:13.660 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~5s 2023-07-08T14:12:13.661 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~5s 2023-07-08T14:12:13.661 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~5s 2023-07-08T14:12:13.661 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~5s 2023-07-08T14:12:13.661 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~5s 2023-07-08T14:12:13.661 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~5s 2023-07-08T14:12:13.661 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~5s 2023-07-08T14:12:13.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~5s 2023-07-08T14:12:13.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~5s 2023-07-08T14:12:13.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~5s 2023-07-08T14:12:13.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~5s 2023-07-08T14:12:13.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~5s 2023-07-08T14:12:13.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~5s 2023-07-08T14:12:13.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~5s 2023-07-08T14:12:13.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~5s 2023-07-08T14:12:13.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~5s 2023-07-08T14:12:13.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~5s 2023-07-08T14:12:15.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:12:15.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:12:18.664 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~10s 2023-07-08T14:12:18.665 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~10s 2023-07-08T14:12:18.665 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~10s 2023-07-08T14:12:18.665 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~10s 2023-07-08T14:12:18.665 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~10s 2023-07-08T14:12:18.665 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~10s 2023-07-08T14:12:18.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~10s 2023-07-08T14:12:18.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~10s 2023-07-08T14:12:18.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~10s 2023-07-08T14:12:18.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~10s 2023-07-08T14:12:18.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~10s 2023-07-08T14:12:18.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~10s 2023-07-08T14:12:18.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~10s 2023-07-08T14:12:18.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~10s 2023-07-08T14:12:18.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~10s 2023-07-08T14:12:18.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~10s 2023-07-08T14:12:18.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~10s 2023-07-08T14:12:23.672 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~15s 2023-07-08T14:12:23.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~15s 2023-07-08T14:12:23.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~15s 2023-07-08T14:12:23.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~15s 2023-07-08T14:12:23.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~15s 2023-07-08T14:12:23.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~15s 2023-07-08T14:12:23.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~15s 2023-07-08T14:12:23.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~15s 2023-07-08T14:12:23.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~15s 2023-07-08T14:12:23.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~15s 2023-07-08T14:12:23.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~15s 2023-07-08T14:12:23.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~15s 2023-07-08T14:12:23.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~15s 2023-07-08T14:12:23.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~15s 2023-07-08T14:12:23.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~15s 2023-07-08T14:12:23.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~15s 2023-07-08T14:12:23.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~15s 2023-07-08T14:12:28.680 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~20s 2023-07-08T14:12:28.681 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~20s 2023-07-08T14:12:28.681 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~20s 2023-07-08T14:12:28.681 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~20s 2023-07-08T14:12:28.681 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~20s 2023-07-08T14:12:28.681 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~20s 2023-07-08T14:12:28.682 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~20s 2023-07-08T14:12:28.682 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~20s 2023-07-08T14:12:28.682 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~20s 2023-07-08T14:12:28.682 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~20s 2023-07-08T14:12:28.682 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~20s 2023-07-08T14:12:28.682 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~20s 2023-07-08T14:12:28.682 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~20s 2023-07-08T14:12:28.683 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~20s 2023-07-08T14:12:28.683 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~20s 2023-07-08T14:12:28.683 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~20s 2023-07-08T14:12:28.683 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~20s 2023-07-08T14:12:29.710 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:12:29.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:12:33.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~25s 2023-07-08T14:12:33.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~25s 2023-07-08T14:12:33.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~25s 2023-07-08T14:12:33.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~25s 2023-07-08T14:12:33.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~25s 2023-07-08T14:12:33.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~25s 2023-07-08T14:12:33.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~25s 2023-07-08T14:12:33.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~25s 2023-07-08T14:12:33.686 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~25s 2023-07-08T14:12:33.686 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~25s 2023-07-08T14:12:33.686 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~25s 2023-07-08T14:12:33.686 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~25s 2023-07-08T14:12:33.686 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~25s 2023-07-08T14:12:33.686 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~25s 2023-07-08T14:12:33.686 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~25s 2023-07-08T14:12:33.687 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~25s 2023-07-08T14:12:33.687 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~25s 2023-07-08T14:12:38.692 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~30s 2023-07-08T14:12:38.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~30s 2023-07-08T14:12:38.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~30s 2023-07-08T14:12:38.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~30s 2023-07-08T14:12:38.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~30s 2023-07-08T14:12:38.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~30s 2023-07-08T14:12:38.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~30s 2023-07-08T14:12:38.694 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~30s 2023-07-08T14:12:38.694 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~30s 2023-07-08T14:12:38.694 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~30s 2023-07-08T14:12:38.694 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~30s 2023-07-08T14:12:38.694 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~30s 2023-07-08T14:12:38.694 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~30s 2023-07-08T14:12:38.694 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~30s 2023-07-08T14:12:38.694 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~30s 2023-07-08T14:12:38.695 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~30s 2023-07-08T14:12:38.695 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~30s 2023-07-08T14:12:43.700 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~35s 2023-07-08T14:12:43.701 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~35s 2023-07-08T14:12:43.701 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~35s 2023-07-08T14:12:43.701 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~35s 2023-07-08T14:12:43.701 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~35s 2023-07-08T14:12:43.701 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~35s 2023-07-08T14:12:43.701 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~35s 2023-07-08T14:12:43.702 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~35s 2023-07-08T14:12:43.702 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~35s 2023-07-08T14:12:43.702 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~35s 2023-07-08T14:12:43.702 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~35s 2023-07-08T14:12:43.702 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~35s 2023-07-08T14:12:43.702 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~35s 2023-07-08T14:12:43.702 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~35s 2023-07-08T14:12:43.702 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~35s 2023-07-08T14:12:43.703 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~35s 2023-07-08T14:12:43.703 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~35s 2023-07-08T14:12:48.116 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:12:48.117 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:12:48.704 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~40s 2023-07-08T14:12:48.704 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~40s 2023-07-08T14:12:48.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~40s 2023-07-08T14:12:48.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~40s 2023-07-08T14:12:48.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~40s 2023-07-08T14:12:48.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~40s 2023-07-08T14:12:48.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~40s 2023-07-08T14:12:48.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~40s 2023-07-08T14:12:48.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~40s 2023-07-08T14:12:48.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~40s 2023-07-08T14:12:48.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~40s 2023-07-08T14:12:48.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~40s 2023-07-08T14:12:48.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~40s 2023-07-08T14:12:48.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~40s 2023-07-08T14:12:48.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~40s 2023-07-08T14:12:48.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~40s 2023-07-08T14:12:48.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~40s 2023-07-08T14:12:53.712 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~45s 2023-07-08T14:12:53.713 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~45s 2023-07-08T14:12:53.713 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~45s 2023-07-08T14:12:53.713 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~45s 2023-07-08T14:12:53.713 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~45s 2023-07-08T14:12:53.713 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~45s 2023-07-08T14:12:53.713 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~45s 2023-07-08T14:12:53.713 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~45s 2023-07-08T14:12:53.714 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~45s 2023-07-08T14:12:53.714 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~45s 2023-07-08T14:12:53.714 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~45s 2023-07-08T14:12:53.714 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~45s 2023-07-08T14:12:53.714 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~45s 2023-07-08T14:12:53.714 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~45s 2023-07-08T14:12:53.714 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~45s 2023-07-08T14:12:53.714 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~45s 2023-07-08T14:12:53.715 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~45s 2023-07-08T14:12:57.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:12:57.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:12:58.717 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~50s 2023-07-08T14:12:58.717 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~50s 2023-07-08T14:12:58.717 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~50s 2023-07-08T14:12:58.717 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~50s 2023-07-08T14:12:58.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~50s 2023-07-08T14:12:58.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~50s 2023-07-08T14:12:58.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~50s 2023-07-08T14:12:58.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~50s 2023-07-08T14:12:58.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~50s 2023-07-08T14:12:58.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~50s 2023-07-08T14:12:58.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~50s 2023-07-08T14:12:58.719 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~50s 2023-07-08T14:12:58.719 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~50s 2023-07-08T14:12:58.719 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~50s 2023-07-08T14:12:58.719 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~50s 2023-07-08T14:12:58.719 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~50s 2023-07-08T14:12:58.719 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~50s 2023-07-08T14:13:00.119 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:13:00.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:13:05.125 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:13:05.126 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~56s 2023-07-08T14:13:05.126 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~56s 2023-07-08T14:13:05.126 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~56s 2023-07-08T14:13:05.126 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~56s 2023-07-08T14:13:05.126 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~56s 2023-07-08T14:13:05.126 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~56s 2023-07-08T14:13:05.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~56s 2023-07-08T14:13:05.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~56s 2023-07-08T14:13:05.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~56s 2023-07-08T14:13:05.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~56s 2023-07-08T14:13:05.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~56s 2023-07-08T14:13:05.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~56s 2023-07-08T14:13:05.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~56s 2023-07-08T14:13:05.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~56s 2023-07-08T14:13:05.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~56s 2023-07-08T14:13:05.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~56s 2023-07-08T14:13:05.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~56s 2023-07-08T14:13:08.176 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:13:10.131 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~61s 2023-07-08T14:13:10.132 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~61s 2023-07-08T14:13:10.132 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~61s 2023-07-08T14:13:10.132 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~61s 2023-07-08T14:13:10.132 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~61s 2023-07-08T14:13:10.132 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~61s 2023-07-08T14:13:10.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~61s 2023-07-08T14:13:10.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~61s 2023-07-08T14:13:10.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~61s 2023-07-08T14:13:10.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~61s 2023-07-08T14:13:10.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~61s 2023-07-08T14:13:10.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~61s 2023-07-08T14:13:10.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~61s 2023-07-08T14:13:10.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~61s 2023-07-08T14:13:10.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~61s 2023-07-08T14:13:10.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~61s 2023-07-08T14:13:10.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~61s 2023-07-08T14:13:11.178 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:13:11.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:13:16.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~68s 2023-07-08T14:13:16.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~68s 2023-07-08T14:13:16.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~68s 2023-07-08T14:13:16.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~68s 2023-07-08T14:13:16.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~68s 2023-07-08T14:13:16.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~68s 2023-07-08T14:13:16.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~68s 2023-07-08T14:13:16.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~68s 2023-07-08T14:13:16.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~68s 2023-07-08T14:13:16.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~68s 2023-07-08T14:13:16.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~68s 2023-07-08T14:13:16.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~68s 2023-07-08T14:13:16.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~68s 2023-07-08T14:13:16.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~68s 2023-07-08T14:13:16.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~68s 2023-07-08T14:13:16.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~68s 2023-07-08T14:13:16.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~68s 2023-07-08T14:13:17.127 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:13:17.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:13:22.133 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:13:22.134 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:13:22.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:13:22.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:13:27.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~73s 2023-07-08T14:13:27.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~73s 2023-07-08T14:13:27.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~73s 2023-07-08T14:13:27.142 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~73s 2023-07-08T14:13:27.142 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~73s 2023-07-08T14:13:27.142 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~73s 2023-07-08T14:13:27.142 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~73s 2023-07-08T14:13:27.142 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~73s 2023-07-08T14:13:27.142 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~73s 2023-07-08T14:13:27.142 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~73s 2023-07-08T14:13:27.143 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~73s 2023-07-08T14:13:27.143 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~73s 2023-07-08T14:13:27.143 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~73s 2023-07-08T14:13:27.143 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~73s 2023-07-08T14:13:27.143 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~73s 2023-07-08T14:13:27.143 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~73s 2023-07-08T14:13:27.143 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~73s 2023-07-08T14:13:27.144 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:13:30.147 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:13:30.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:13:40.156 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:13:40.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~92s 2023-07-08T14:13:40.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~92s 2023-07-08T14:13:40.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~92s 2023-07-08T14:13:40.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~92s 2023-07-08T14:13:40.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~92s 2023-07-08T14:13:40.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~92s 2023-07-08T14:13:40.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~92s 2023-07-08T14:13:40.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~92s 2023-07-08T14:13:40.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~92s 2023-07-08T14:13:40.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~92s 2023-07-08T14:13:40.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~92s 2023-07-08T14:13:40.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~92s 2023-07-08T14:13:40.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~92s 2023-07-08T14:13:40.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~92s 2023-07-08T14:13:40.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~92s 2023-07-08T14:13:40.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~92s 2023-07-08T14:13:40.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~92s 2023-07-08T14:13:43.216 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:13:43.217 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:13:43.221 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 183, in invoke_logrotate procs = ctx.cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:13:45.162 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~97s 2023-07-08T14:13:45.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~97s 2023-07-08T14:13:45.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~97s 2023-07-08T14:13:45.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~97s 2023-07-08T14:13:45.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~97s 2023-07-08T14:13:45.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~97s 2023-07-08T14:13:45.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~97s 2023-07-08T14:13:45.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~97s 2023-07-08T14:13:45.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~97s 2023-07-08T14:13:45.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~97s 2023-07-08T14:13:45.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~97s 2023-07-08T14:13:45.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~97s 2023-07-08T14:13:45.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~97s 2023-07-08T14:13:45.165 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~97s 2023-07-08T14:13:45.165 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~97s 2023-07-08T14:13:45.165 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~97s 2023-07-08T14:13:45.165 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~97s 2023-07-08T14:13:49.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:13:49.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:13:54.168 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~106s 2023-07-08T14:13:54.168 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~106s 2023-07-08T14:13:54.168 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~106s 2023-07-08T14:13:54.168 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~106s 2023-07-08T14:13:54.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~106s 2023-07-08T14:13:54.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~106s 2023-07-08T14:13:54.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~106s 2023-07-08T14:13:54.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~106s 2023-07-08T14:13:54.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~106s 2023-07-08T14:13:54.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~106s 2023-07-08T14:13:54.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~106s 2023-07-08T14:13:54.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~106s 2023-07-08T14:13:54.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~106s 2023-07-08T14:13:54.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~106s 2023-07-08T14:13:54.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~106s 2023-07-08T14:13:54.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~106s 2023-07-08T14:13:54.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~106s 2023-07-08T14:13:57.232 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:13:59.172 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~111s 2023-07-08T14:13:59.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~111s 2023-07-08T14:13:59.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~111s 2023-07-08T14:13:59.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~111s 2023-07-08T14:13:59.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~111s 2023-07-08T14:13:59.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~111s 2023-07-08T14:13:59.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~111s 2023-07-08T14:13:59.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~111s 2023-07-08T14:13:59.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~111s 2023-07-08T14:13:59.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~111s 2023-07-08T14:13:59.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~111s 2023-07-08T14:13:59.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~111s 2023-07-08T14:13:59.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~111s 2023-07-08T14:13:59.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~111s 2023-07-08T14:13:59.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~111s 2023-07-08T14:13:59.175 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~111s 2023-07-08T14:13:59.175 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~111s 2023-07-08T14:14:04.180 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~116s 2023-07-08T14:14:04.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~116s 2023-07-08T14:14:04.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~116s 2023-07-08T14:14:04.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~116s 2023-07-08T14:14:04.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~116s 2023-07-08T14:14:04.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~116s 2023-07-08T14:14:04.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~116s 2023-07-08T14:14:04.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~116s 2023-07-08T14:14:04.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~116s 2023-07-08T14:14:04.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~116s 2023-07-08T14:14:04.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~116s 2023-07-08T14:14:04.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~116s 2023-07-08T14:14:04.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~116s 2023-07-08T14:14:04.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~116s 2023-07-08T14:14:04.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~116s 2023-07-08T14:14:04.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~116s 2023-07-08T14:14:04.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~116s 2023-07-08T14:14:09.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~121s 2023-07-08T14:14:09.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~121s 2023-07-08T14:14:09.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~121s 2023-07-08T14:14:09.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~121s 2023-07-08T14:14:09.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~121s 2023-07-08T14:14:09.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~121s 2023-07-08T14:14:09.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~121s 2023-07-08T14:14:09.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~121s 2023-07-08T14:14:09.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~121s 2023-07-08T14:14:09.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~121s 2023-07-08T14:14:09.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~121s 2023-07-08T14:14:09.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~121s 2023-07-08T14:14:09.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~121s 2023-07-08T14:14:09.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~121s 2023-07-08T14:14:09.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~121s 2023-07-08T14:14:09.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~121s 2023-07-08T14:14:09.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~121s 2023-07-08T14:14:09.233 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:14:09.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:14:14.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~126s 2023-07-08T14:14:14.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~126s 2023-07-08T14:14:14.240 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~126s 2023-07-08T14:14:14.240 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~126s 2023-07-08T14:14:14.240 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~126s 2023-07-08T14:14:14.240 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~126s 2023-07-08T14:14:14.240 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~126s 2023-07-08T14:14:14.240 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~126s 2023-07-08T14:14:14.240 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~126s 2023-07-08T14:14:14.241 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~126s 2023-07-08T14:14:14.241 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~126s 2023-07-08T14:14:14.241 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~126s 2023-07-08T14:14:14.241 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~126s 2023-07-08T14:14:14.241 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~126s 2023-07-08T14:14:14.241 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~126s 2023-07-08T14:14:14.241 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~126s 2023-07-08T14:14:14.242 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~126s 2023-07-08T14:14:17.296 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:14:19.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~131s 2023-07-08T14:14:19.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~131s 2023-07-08T14:14:19.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~131s 2023-07-08T14:14:19.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~131s 2023-07-08T14:14:19.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~131s 2023-07-08T14:14:19.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~131s 2023-07-08T14:14:19.247 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~131s 2023-07-08T14:14:19.247 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~131s 2023-07-08T14:14:19.247 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~131s 2023-07-08T14:14:19.247 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~131s 2023-07-08T14:14:19.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~131s 2023-07-08T14:14:19.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~131s 2023-07-08T14:14:19.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~131s 2023-07-08T14:14:19.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~131s 2023-07-08T14:14:19.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~131s 2023-07-08T14:14:19.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~131s 2023-07-08T14:14:19.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~131s 2023-07-08T14:14:20.298 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:14:20.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:14:20.372 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:14:20.373 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:14:20.373 INFO:tasks.cephfs_test_runner:test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) ... ERROR 2023-07-08T14:14:20.374 INFO:tasks.cephfs_test_runner:ERROR 2023-07-08T14:14:20.374 INFO:tasks.cephfs_test_runner: 2023-07-08T14:14:20.375 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-08T14:14:20.375 INFO:tasks.cephfs_test_runner:ERROR: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-07-08T14:14:20.375 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-08T14:14:20.375 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-08T14:14:20.375 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/tests_from_xfstests_dev.py", line 12, in test_generic 2023-07-08T14:14:20.376 INFO:tasks.cephfs_test_runner: self.run_generic_tests() 2023-07-08T14:14:20.376 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/xfstests_dev.py", line 387, in run_generic_tests 2023-07-08T14:14:20.376 INFO:tasks.cephfs_test_runner: return self.run_testdir('generic') 2023-07-08T14:14:20.377 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/xfstests_dev.py", line 373, in run_testdir 2023-07-08T14:14:20.377 INFO:tasks.cephfs_test_runner: self.run_testfile(testdir, testfile) 2023-07-08T14:14:20.377 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/xfstests_dev.py", line 363, in run_testfile 2023-07-08T14:14:20.377 INFO:tasks.cephfs_test_runner: return self.run_test(f'{testdir}/{testfile}', exit_on_err) 2023-07-08T14:14:20.377 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/xfstests_dev.py", line 337, in run_test 2023-07-08T14:14:20.378 INFO:tasks.cephfs_test_runner: p = self.mount_a.run_shell(args=cmd, cwd=self.xfstests_repo_path, 2023-07-08T14:14:20.378 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-07-08T14:14:20.378 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-07-08T14:14:20.378 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run 2023-07-08T14:14:20.378 INFO:tasks.cephfs_test_runner: raise Exception(f'Cannot connect to remote host {self.shortname}') 2023-07-08T14:14:20.379 INFO:tasks.cephfs_test_runner:Exception: Cannot connect to remote host smithi012 2023-07-08T14:14:20.379 INFO:tasks.cephfs_test_runner: 2023-07-08T14:14:20.379 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-08T14:14:20.379 INFO:tasks.cephfs_test_runner:ERROR: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-07-08T14:14:20.379 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-08T14:14:20.380 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-08T14:14:20.380 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/xfstests_dev.py", line 51, in tearDown 2023-07-08T14:14:20.380 INFO:tasks.cephfs_test_runner: self.del_users_and_groups() 2023-07-08T14:14:20.380 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/xfstests_dev.py", line 56, in del_users_and_groups 2023-07-08T14:14:20.380 INFO:tasks.cephfs_test_runner: self.mount_a.client_remote.run(args=['sudo', 'userdel', '--force', 2023-07-08T14:14:20.381 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run 2023-07-08T14:14:20.381 INFO:tasks.cephfs_test_runner: raise Exception(f'Cannot connect to remote host {self.shortname}') 2023-07-08T14:14:20.381 INFO:tasks.cephfs_test_runner:Exception: Cannot connect to remote host smithi012 2023-07-08T14:14:20.381 INFO:tasks.cephfs_test_runner: 2023-07-08T14:14:20.381 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-08T14:14:20.382 INFO:tasks.cephfs_test_runner:Ran 1 test in 5717.496s 2023-07-08T14:14:20.382 INFO:tasks.cephfs_test_runner: 2023-07-08T14:14:20.382 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2023-07-08T14:14:20.382 INFO:tasks.cephfs_test_runner: 2023-07-08T14:14:20.382 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-08T14:14:20.383 INFO:tasks.cephfs_test_runner:ERROR: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-07-08T14:14:20.383 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-08T14:14:20.383 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-08T14:14:20.383 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/tests_from_xfstests_dev.py", line 12, in test_generic 2023-07-08T14:14:20.383 INFO:tasks.cephfs_test_runner: self.run_generic_tests() 2023-07-08T14:14:20.384 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/xfstests_dev.py", line 387, in run_generic_tests 2023-07-08T14:14:20.384 INFO:tasks.cephfs_test_runner: return self.run_testdir('generic') 2023-07-08T14:14:20.384 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/xfstests_dev.py", line 373, in run_testdir 2023-07-08T14:14:20.385 INFO:tasks.cephfs_test_runner: self.run_testfile(testdir, testfile) 2023-07-08T14:14:20.385 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/xfstests_dev.py", line 363, in run_testfile 2023-07-08T14:14:20.385 INFO:tasks.cephfs_test_runner: return self.run_test(f'{testdir}/{testfile}', exit_on_err) 2023-07-08T14:14:20.385 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/xfstests_dev.py", line 337, in run_test 2023-07-08T14:14:20.385 INFO:tasks.cephfs_test_runner: p = self.mount_a.run_shell(args=cmd, cwd=self.xfstests_repo_path, 2023-07-08T14:14:20.385 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-07-08T14:14:20.386 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-07-08T14:14:20.386 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run 2023-07-08T14:14:20.387 INFO:tasks.cephfs_test_runner: raise Exception(f'Cannot connect to remote host {self.shortname}') 2023-07-08T14:14:20.387 INFO:tasks.cephfs_test_runner:Exception: Cannot connect to remote host smithi012 2023-07-08T14:14:20.387 INFO:tasks.cephfs_test_runner: 2023-07-08T14:14:20.387 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-08T14:14:20.388 INFO:tasks.cephfs_test_runner:ERROR: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-07-08T14:14:20.388 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-08T14:14:20.388 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-08T14:14:20.388 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/xfstests_dev.py", line 51, in tearDown 2023-07-08T14:14:20.388 INFO:tasks.cephfs_test_runner: self.del_users_and_groups() 2023-07-08T14:14:20.389 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/xfstests_dev.py", line 56, in del_users_and_groups 2023-07-08T14:14:20.389 INFO:tasks.cephfs_test_runner: self.mount_a.client_remote.run(args=['sudo', 'userdel', '--force', 2023-07-08T14:14:20.389 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run 2023-07-08T14:14:20.389 INFO:tasks.cephfs_test_runner: raise Exception(f'Cannot connect to remote host {self.shortname}') 2023-07-08T14:14:20.389 INFO:tasks.cephfs_test_runner:Exception: Cannot connect to remote host smithi012 2023-07-08T14:14:20.390 INFO:tasks.cephfs_test_runner: 2023-07-08T14:14:20.390 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev), test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-07-08T14:14:20.453 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=259cc0e229f647f48bbd8274a36b258b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev), test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-07-08T14:14:20.455 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-07-08T14:14:20.464 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-08T14:14:20.501 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-08T14:14:20.502 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:14:20.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:14:23.572 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:14:24.251 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~136s 2023-07-08T14:14:24.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~136s 2023-07-08T14:14:24.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~136s 2023-07-08T14:14:24.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~136s 2023-07-08T14:14:24.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~136s 2023-07-08T14:14:24.253 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~136s 2023-07-08T14:14:24.253 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~136s 2023-07-08T14:14:24.253 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~136s 2023-07-08T14:14:24.253 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~136s 2023-07-08T14:14:24.253 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~136s 2023-07-08T14:14:24.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~136s 2023-07-08T14:14:24.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~136s 2023-07-08T14:14:24.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~136s 2023-07-08T14:14:24.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~136s 2023-07-08T14:14:24.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~136s 2023-07-08T14:14:24.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~136s 2023-07-08T14:14:24.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~136s 2023-07-08T14:14:29.261 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~141s 2023-07-08T14:14:29.261 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~141s 2023-07-08T14:14:29.262 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~141s 2023-07-08T14:14:29.262 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~141s 2023-07-08T14:14:29.263 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~141s 2023-07-08T14:14:29.263 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~141s 2023-07-08T14:14:29.263 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~141s 2023-07-08T14:14:29.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~141s 2023-07-08T14:14:29.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~141s 2023-07-08T14:14:29.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~141s 2023-07-08T14:14:29.265 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~141s 2023-07-08T14:14:29.265 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~141s 2023-07-08T14:14:29.266 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~141s 2023-07-08T14:14:29.266 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~141s 2023-07-08T14:14:29.266 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~141s 2023-07-08T14:14:29.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~141s 2023-07-08T14:14:29.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~141s 2023-07-08T14:14:32.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:14:32.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:14:37.582 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~149s 2023-07-08T14:14:37.582 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~149s 2023-07-08T14:14:37.583 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~149s 2023-07-08T14:14:37.583 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~149s 2023-07-08T14:14:37.584 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~149s 2023-07-08T14:14:37.584 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~149s 2023-07-08T14:14:37.584 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~149s 2023-07-08T14:14:37.585 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~149s 2023-07-08T14:14:37.585 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~149s 2023-07-08T14:14:37.586 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~149s 2023-07-08T14:14:37.586 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~149s 2023-07-08T14:14:37.586 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~149s 2023-07-08T14:14:37.587 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~149s 2023-07-08T14:14:37.587 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~149s 2023-07-08T14:14:37.588 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~149s 2023-07-08T14:14:37.588 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~149s 2023-07-08T14:14:37.588 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~149s 2023-07-08T14:14:40.656 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:14:42.591 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~154s 2023-07-08T14:14:42.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~154s 2023-07-08T14:14:42.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~154s 2023-07-08T14:14:42.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~154s 2023-07-08T14:14:42.593 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~154s 2023-07-08T14:14:42.593 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~154s 2023-07-08T14:14:42.593 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~154s 2023-07-08T14:14:42.593 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~154s 2023-07-08T14:14:42.593 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~154s 2023-07-08T14:14:42.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~154s 2023-07-08T14:14:42.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~154s 2023-07-08T14:14:42.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~154s 2023-07-08T14:14:42.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~154s 2023-07-08T14:14:42.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~154s 2023-07-08T14:14:42.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~154s 2023-07-08T14:14:42.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~154s 2023-07-08T14:14:42.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~154s 2023-07-08T14:14:47.600 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~159s 2023-07-08T14:14:47.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~159s 2023-07-08T14:14:47.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~159s 2023-07-08T14:14:47.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~159s 2023-07-08T14:14:47.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~159s 2023-07-08T14:14:47.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~159s 2023-07-08T14:14:47.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~159s 2023-07-08T14:14:47.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~159s 2023-07-08T14:14:47.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~159s 2023-07-08T14:14:47.603 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~159s 2023-07-08T14:14:47.603 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~159s 2023-07-08T14:14:47.603 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~159s 2023-07-08T14:14:47.603 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~159s 2023-07-08T14:14:47.603 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~159s 2023-07-08T14:14:47.604 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~159s 2023-07-08T14:14:47.604 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~159s 2023-07-08T14:14:47.604 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~159s 2023-07-08T14:14:52.608 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~164s 2023-07-08T14:14:52.609 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~164s 2023-07-08T14:14:52.609 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~164s 2023-07-08T14:14:52.609 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~164s 2023-07-08T14:14:52.609 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~164s 2023-07-08T14:14:52.610 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~164s 2023-07-08T14:14:52.610 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~164s 2023-07-08T14:14:52.610 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~164s 2023-07-08T14:14:52.610 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~164s 2023-07-08T14:14:52.610 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~164s 2023-07-08T14:14:52.611 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~164s 2023-07-08T14:14:52.611 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~164s 2023-07-08T14:14:52.611 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~164s 2023-07-08T14:14:52.611 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~164s 2023-07-08T14:14:52.612 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~164s 2023-07-08T14:14:52.612 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~164s 2023-07-08T14:14:52.612 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~164s 2023-07-08T14:14:52.658 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:14:52.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:14:57.665 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~169s 2023-07-08T14:14:57.665 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~169s 2023-07-08T14:14:57.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~169s 2023-07-08T14:14:57.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~169s 2023-07-08T14:14:57.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~169s 2023-07-08T14:14:57.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~169s 2023-07-08T14:14:57.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~169s 2023-07-08T14:14:57.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~169s 2023-07-08T14:14:57.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~169s 2023-07-08T14:14:57.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~169s 2023-07-08T14:14:57.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~169s 2023-07-08T14:14:57.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~169s 2023-07-08T14:14:57.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~169s 2023-07-08T14:14:57.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~169s 2023-07-08T14:14:57.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~169s 2023-07-08T14:14:57.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~169s 2023-07-08T14:14:57.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~169s 2023-07-08T14:15:00.724 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:15:02.672 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~174s 2023-07-08T14:15:02.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~174s 2023-07-08T14:15:02.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~174s 2023-07-08T14:15:02.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~174s 2023-07-08T14:15:02.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~174s 2023-07-08T14:15:02.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~174s 2023-07-08T14:15:02.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~174s 2023-07-08T14:15:02.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~174s 2023-07-08T14:15:02.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~174s 2023-07-08T14:15:02.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~174s 2023-07-08T14:15:02.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~174s 2023-07-08T14:15:02.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~174s 2023-07-08T14:15:02.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~174s 2023-07-08T14:15:02.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~174s 2023-07-08T14:15:02.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~174s 2023-07-08T14:15:02.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~174s 2023-07-08T14:15:02.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~174s 2023-07-08T14:15:03.726 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:15:03.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:15:08.733 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~180s 2023-07-08T14:15:08.734 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~180s 2023-07-08T14:15:08.734 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~180s 2023-07-08T14:15:08.734 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~180s 2023-07-08T14:15:08.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~180s 2023-07-08T14:15:08.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~180s 2023-07-08T14:15:08.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~180s 2023-07-08T14:15:08.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~180s 2023-07-08T14:15:08.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~180s 2023-07-08T14:15:08.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~180s 2023-07-08T14:15:08.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~180s 2023-07-08T14:15:08.737 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~180s 2023-07-08T14:15:08.737 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~180s 2023-07-08T14:15:08.737 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~180s 2023-07-08T14:15:08.738 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~180s 2023-07-08T14:15:08.738 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~180s 2023-07-08T14:15:08.738 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~180s 2023-07-08T14:15:11.792 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:15:11.793 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:15:11.793 ERROR:teuthology.run_tasks:Manager failed: kclient Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:15:11.798 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-07-08T14:15:11.807 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-08T14:15:11.808 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:15:11.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:15:13.740 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~185s 2023-07-08T14:15:13.740 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~185s 2023-07-08T14:15:13.740 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~185s 2023-07-08T14:15:13.741 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~185s 2023-07-08T14:15:13.741 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~185s 2023-07-08T14:15:13.741 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~185s 2023-07-08T14:15:13.741 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~185s 2023-07-08T14:15:13.742 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~185s 2023-07-08T14:15:13.742 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~185s 2023-07-08T14:15:13.742 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~185s 2023-07-08T14:15:13.742 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~185s 2023-07-08T14:15:13.742 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~185s 2023-07-08T14:15:13.742 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~185s 2023-07-08T14:15:13.743 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~185s 2023-07-08T14:15:13.743 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~185s 2023-07-08T14:15:13.743 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~185s 2023-07-08T14:15:13.743 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~185s 2023-07-08T14:15:14.864 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:15:18.748 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~190s 2023-07-08T14:15:18.749 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~190s 2023-07-08T14:15:18.749 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~190s 2023-07-08T14:15:18.750 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~190s 2023-07-08T14:15:18.750 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~190s 2023-07-08T14:15:18.750 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~190s 2023-07-08T14:15:18.750 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~190s 2023-07-08T14:15:18.750 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~190s 2023-07-08T14:15:18.751 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~190s 2023-07-08T14:15:18.751 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~190s 2023-07-08T14:15:18.751 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~190s 2023-07-08T14:15:18.751 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~190s 2023-07-08T14:15:18.751 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~190s 2023-07-08T14:15:18.752 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~190s 2023-07-08T14:15:18.752 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~190s 2023-07-08T14:15:18.752 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~190s 2023-07-08T14:15:18.752 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~190s 2023-07-08T14:15:23.756 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~195s 2023-07-08T14:15:23.757 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~195s 2023-07-08T14:15:23.757 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~195s 2023-07-08T14:15:23.758 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~195s 2023-07-08T14:15:23.758 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~195s 2023-07-08T14:15:23.758 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~195s 2023-07-08T14:15:23.758 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~195s 2023-07-08T14:15:23.758 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~195s 2023-07-08T14:15:23.759 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~195s 2023-07-08T14:15:23.759 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~195s 2023-07-08T14:15:23.759 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~195s 2023-07-08T14:15:23.759 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~195s 2023-07-08T14:15:23.760 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~195s 2023-07-08T14:15:23.760 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~195s 2023-07-08T14:15:23.760 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~195s 2023-07-08T14:15:23.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~195s 2023-07-08T14:15:23.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~195s 2023-07-08T14:15:23.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:15:23.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:15:28.874 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~200s 2023-07-08T14:15:28.874 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~200s 2023-07-08T14:15:28.875 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~200s 2023-07-08T14:15:28.875 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~200s 2023-07-08T14:15:28.876 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~200s 2023-07-08T14:15:28.877 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~200s 2023-07-08T14:15:28.877 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~200s 2023-07-08T14:15:28.877 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~200s 2023-07-08T14:15:28.878 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~200s 2023-07-08T14:15:28.878 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~200s 2023-07-08T14:15:28.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~200s 2023-07-08T14:15:28.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~200s 2023-07-08T14:15:28.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~200s 2023-07-08T14:15:28.880 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~200s 2023-07-08T14:15:28.880 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~200s 2023-07-08T14:15:28.880 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~200s 2023-07-08T14:15:28.881 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~200s 2023-07-08T14:15:31.952 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:15:33.884 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~205s 2023-07-08T14:15:33.885 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~205s 2023-07-08T14:15:33.885 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~205s 2023-07-08T14:15:33.885 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~205s 2023-07-08T14:15:33.885 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~205s 2023-07-08T14:15:33.886 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~205s 2023-07-08T14:15:33.886 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~205s 2023-07-08T14:15:33.886 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~205s 2023-07-08T14:15:33.886 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~205s 2023-07-08T14:15:33.886 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~205s 2023-07-08T14:15:33.887 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~205s 2023-07-08T14:15:33.887 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~205s 2023-07-08T14:15:33.887 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~205s 2023-07-08T14:15:33.887 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~205s 2023-07-08T14:15:33.887 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~205s 2023-07-08T14:15:33.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~205s 2023-07-08T14:15:33.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~205s 2023-07-08T14:15:38.893 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~210s 2023-07-08T14:15:38.893 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~210s 2023-07-08T14:15:38.894 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~210s 2023-07-08T14:15:38.894 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~210s 2023-07-08T14:15:38.895 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~210s 2023-07-08T14:15:38.895 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~210s 2023-07-08T14:15:38.895 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~210s 2023-07-08T14:15:38.896 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~210s 2023-07-08T14:15:38.896 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~210s 2023-07-08T14:15:38.897 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~210s 2023-07-08T14:15:38.897 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~210s 2023-07-08T14:15:38.898 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~210s 2023-07-08T14:15:38.898 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~210s 2023-07-08T14:15:38.898 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~210s 2023-07-08T14:15:38.899 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~210s 2023-07-08T14:15:38.899 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~210s 2023-07-08T14:15:38.900 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~210s 2023-07-08T14:15:43.904 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~215s 2023-07-08T14:15:43.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~215s 2023-07-08T14:15:43.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~215s 2023-07-08T14:15:43.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~215s 2023-07-08T14:15:43.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~215s 2023-07-08T14:15:43.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~215s 2023-07-08T14:15:43.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~215s 2023-07-08T14:15:43.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~215s 2023-07-08T14:15:43.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~215s 2023-07-08T14:15:43.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~215s 2023-07-08T14:15:43.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~215s 2023-07-08T14:15:43.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~215s 2023-07-08T14:15:43.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~215s 2023-07-08T14:15:43.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~215s 2023-07-08T14:15:43.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~215s 2023-07-08T14:15:43.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~215s 2023-07-08T14:15:43.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~215s 2023-07-08T14:15:43.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:15:43.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:15:47.668 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:15:48.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~220s 2023-07-08T14:15:48.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~220s 2023-07-08T14:15:48.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~220s 2023-07-08T14:15:48.912 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~220s 2023-07-08T14:15:48.912 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~220s 2023-07-08T14:15:48.912 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~220s 2023-07-08T14:15:48.912 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~220s 2023-07-08T14:15:48.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~220s 2023-07-08T14:15:48.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~220s 2023-07-08T14:15:48.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~220s 2023-07-08T14:15:48.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~220s 2023-07-08T14:15:48.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~220s 2023-07-08T14:15:48.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~220s 2023-07-08T14:15:48.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~220s 2023-07-08T14:15:48.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~220s 2023-07-08T14:15:48.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~220s 2023-07-08T14:15:48.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~220s 2023-07-08T14:15:50.671 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:15:50.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:15:55.677 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~227s 2023-07-08T14:15:55.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~227s 2023-07-08T14:15:55.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~227s 2023-07-08T14:15:55.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~227s 2023-07-08T14:15:55.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~227s 2023-07-08T14:15:55.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~227s 2023-07-08T14:15:55.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~227s 2023-07-08T14:15:55.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~227s 2023-07-08T14:15:55.680 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~227s 2023-07-08T14:15:55.680 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~227s 2023-07-08T14:15:55.680 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~227s 2023-07-08T14:15:55.680 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~227s 2023-07-08T14:15:55.680 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~227s 2023-07-08T14:15:55.681 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~227s 2023-07-08T14:15:55.681 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~227s 2023-07-08T14:15:55.681 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~227s 2023-07-08T14:15:55.681 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~227s 2023-07-08T14:15:58.736 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:15:58.737 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:15:58.738 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:15:58.749 INFO:teuthology.misc:Shutting down mds daemons... 2023-07-08T14:15:58.750 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-08T14:15:58.751 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:58.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:58.755 ERROR:teuthology.misc:Saw exception from ceph.mds.a Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi012 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-07-08T14:15:58.768 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-08T14:15:58.768 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:58.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:58.769 ERROR:teuthology.misc:Saw exception from ceph.mds.c Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi012 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2023-07-08T14:15:58.782 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-08T14:15:58.782 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:58.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:58.784 ERROR:teuthology.misc:Saw exception from ceph.mds.b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-07-08T14:15:58.796 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-08T14:15:58.797 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:58.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:58.798 ERROR:teuthology.misc:Saw exception from ceph.mds.d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2023-07-08T14:15:58.810 INFO:teuthology.misc:Shutting down osd daemons... 2023-07-08T14:15:58.811 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-07-08T14:15:58.811 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:58.812 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:58.812 ERROR:teuthology.misc:Saw exception from ceph.osd.0 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi012 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0' 2023-07-08T14:15:58.828 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-07-08T14:15:58.829 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:58.830 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:58.830 ERROR:teuthology.misc:Saw exception from ceph.osd.1 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi012 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1' 2023-07-08T14:15:58.904 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-07-08T14:15:58.904 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:58.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:58.906 ERROR:teuthology.misc:Saw exception from ceph.osd.2 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi012 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2' 2023-07-08T14:15:58.923 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-07-08T14:15:58.923 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:58.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:58.925 ERROR:teuthology.misc:Saw exception from ceph.osd.3 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi012 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3' 2023-07-08T14:15:58.941 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-07-08T14:15:58.942 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:58.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:58.943 ERROR:teuthology.misc:Saw exception from ceph.osd.4 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4' 2023-07-08T14:15:58.960 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-07-08T14:15:58.960 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:58.961 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:58.962 ERROR:teuthology.misc:Saw exception from ceph.osd.5 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5' 2023-07-08T14:15:58.978 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-07-08T14:15:58.979 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:58.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:58.981 ERROR:teuthology.misc:Saw exception from ceph.osd.6 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6' 2023-07-08T14:15:58.997 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-07-08T14:15:58.998 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:58.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:58.999 ERROR:teuthology.misc:Saw exception from ceph.osd.7 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7' 2023-07-08T14:15:59.016 INFO:teuthology.misc:Shutting down mgr daemons... 2023-07-08T14:15:59.017 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-07-08T14:15:59.018 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:59.019 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:59.019 ERROR:teuthology.misc:Saw exception from ceph.mgr.y Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi012 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y' 2023-07-08T14:15:59.040 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-07-08T14:15:59.041 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:59.042 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:59.042 ERROR:teuthology.misc:Saw exception from ceph.mgr.x Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x' 2023-07-08T14:15:59.065 INFO:teuthology.misc:Shutting down mon daemons... 2023-07-08T14:15:59.066 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-07-08T14:15:59.066 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:59.067 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:59.068 ERROR:teuthology.misc:Saw exception from ceph.mon.a Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi012 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a' 2023-07-08T14:15:59.092 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-07-08T14:15:59.093 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:59.093 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:59.094 ERROR:teuthology.misc:Saw exception from ceph.mon.b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b' 2023-07-08T14:15:59.119 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-07-08T14:15:59.119 INFO:teuthology.orchestra.run:waiting for 300 2023-07-08T14:15:59.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-08T14:15:59.120 ERROR:teuthology.misc:Saw exception from ceph.mon.c Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1471, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1177, in stop_daemons_of_type raise exc File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 1170, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi160 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c' 2023-07-08T14:15:59.145 INFO:tasks.ceph:Checking cluster log for badness... 2023-07-08T14:15:59.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:15:59.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:16:01.002 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~232s 2023-07-08T14:16:01.003 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~232s 2023-07-08T14:16:01.003 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~232s 2023-07-08T14:16:01.003 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~232s 2023-07-08T14:16:01.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~232s 2023-07-08T14:16:01.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~232s 2023-07-08T14:16:01.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~232s 2023-07-08T14:16:01.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~232s 2023-07-08T14:16:01.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~232s 2023-07-08T14:16:01.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~232s 2023-07-08T14:16:01.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~232s 2023-07-08T14:16:01.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~232s 2023-07-08T14:16:01.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~232s 2023-07-08T14:16:01.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~232s 2023-07-08T14:16:01.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~232s 2023-07-08T14:16:01.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~232s 2023-07-08T14:16:01.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~232s 2023-07-08T14:16:01.812 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:16:06.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~237s 2023-07-08T14:16:06.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~237s 2023-07-08T14:16:06.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~237s 2023-07-08T14:16:06.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~237s 2023-07-08T14:16:06.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~237s 2023-07-08T14:16:06.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~237s 2023-07-08T14:16:06.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~237s 2023-07-08T14:16:06.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~237s 2023-07-08T14:16:06.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~237s 2023-07-08T14:16:06.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~237s 2023-07-08T14:16:06.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~237s 2023-07-08T14:16:06.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~237s 2023-07-08T14:16:06.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~237s 2023-07-08T14:16:06.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~237s 2023-07-08T14:16:06.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~237s 2023-07-08T14:16:06.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~237s 2023-07-08T14:16:06.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~237s 2023-07-08T14:16:10.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:16:10.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:16:15.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~247s 2023-07-08T14:16:15.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~247s 2023-07-08T14:16:15.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~247s 2023-07-08T14:16:15.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~247s 2023-07-08T14:16:15.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~247s 2023-07-08T14:16:15.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~247s 2023-07-08T14:16:15.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~247s 2023-07-08T14:16:15.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~247s 2023-07-08T14:16:15.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~247s 2023-07-08T14:16:15.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~247s 2023-07-08T14:16:15.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~247s 2023-07-08T14:16:15.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~247s 2023-07-08T14:16:15.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~247s 2023-07-08T14:16:15.828 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~247s 2023-07-08T14:16:15.828 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~247s 2023-07-08T14:16:15.828 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~247s 2023-07-08T14:16:15.828 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~247s 2023-07-08T14:16:18.896 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:16:20.832 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~252s 2023-07-08T14:16:20.832 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~252s 2023-07-08T14:16:20.832 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~252s 2023-07-08T14:16:20.833 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~252s 2023-07-08T14:16:20.833 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~252s 2023-07-08T14:16:20.833 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~252s 2023-07-08T14:16:20.833 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~252s 2023-07-08T14:16:20.834 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~252s 2023-07-08T14:16:20.834 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~252s 2023-07-08T14:16:20.834 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~252s 2023-07-08T14:16:20.835 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~252s 2023-07-08T14:16:20.835 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~252s 2023-07-08T14:16:20.836 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~252s 2023-07-08T14:16:20.836 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~252s 2023-07-08T14:16:20.837 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~252s 2023-07-08T14:16:20.837 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~252s 2023-07-08T14:16:20.838 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~252s 2023-07-08T14:16:25.841 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~257s 2023-07-08T14:16:25.841 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~257s 2023-07-08T14:16:25.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~257s 2023-07-08T14:16:25.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~257s 2023-07-08T14:16:25.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~257s 2023-07-08T14:16:25.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~257s 2023-07-08T14:16:25.843 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~257s 2023-07-08T14:16:25.843 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~257s 2023-07-08T14:16:25.843 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~257s 2023-07-08T14:16:25.843 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~257s 2023-07-08T14:16:25.843 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~257s 2023-07-08T14:16:25.844 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~257s 2023-07-08T14:16:25.844 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~257s 2023-07-08T14:16:25.844 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~257s 2023-07-08T14:16:25.844 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~257s 2023-07-08T14:16:25.844 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~257s 2023-07-08T14:16:25.845 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~257s 2023-07-08T14:16:30.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~262s 2023-07-08T14:16:30.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~262s 2023-07-08T14:16:30.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~262s 2023-07-08T14:16:30.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~262s 2023-07-08T14:16:30.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~262s 2023-07-08T14:16:30.850 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~262s 2023-07-08T14:16:30.850 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~262s 2023-07-08T14:16:30.850 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~262s 2023-07-08T14:16:30.850 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~262s 2023-07-08T14:16:30.850 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~262s 2023-07-08T14:16:30.851 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~262s 2023-07-08T14:16:30.851 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~262s 2023-07-08T14:16:30.851 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~262s 2023-07-08T14:16:30.851 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~262s 2023-07-08T14:16:30.851 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~262s 2023-07-08T14:16:30.852 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~262s 2023-07-08T14:16:30.852 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~262s 2023-07-08T14:16:30.898 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:16:30.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:16:35.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~267s 2023-07-08T14:16:35.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~267s 2023-07-08T14:16:35.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~267s 2023-07-08T14:16:35.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~267s 2023-07-08T14:16:35.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~267s 2023-07-08T14:16:35.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~267s 2023-07-08T14:16:35.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~267s 2023-07-08T14:16:35.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~267s 2023-07-08T14:16:35.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~267s 2023-07-08T14:16:35.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~267s 2023-07-08T14:16:35.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~267s 2023-07-08T14:16:35.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~267s 2023-07-08T14:16:35.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~267s 2023-07-08T14:16:35.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~267s 2023-07-08T14:16:35.909 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~267s 2023-07-08T14:16:35.909 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~267s 2023-07-08T14:16:35.910 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~267s 2023-07-08T14:16:38.960 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:16:40.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~272s 2023-07-08T14:16:40.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~272s 2023-07-08T14:16:40.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~272s 2023-07-08T14:16:40.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~272s 2023-07-08T14:16:40.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~272s 2023-07-08T14:16:40.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~272s 2023-07-08T14:16:40.916 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~272s 2023-07-08T14:16:40.916 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~272s 2023-07-08T14:16:40.916 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~272s 2023-07-08T14:16:40.917 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~272s 2023-07-08T14:16:40.917 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~272s 2023-07-08T14:16:40.918 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~272s 2023-07-08T14:16:40.918 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~272s 2023-07-08T14:16:40.918 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~272s 2023-07-08T14:16:40.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~272s 2023-07-08T14:16:40.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~272s 2023-07-08T14:16:40.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~272s 2023-07-08T14:16:41.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:16:41.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:16:42.036 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:16:42.037 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:16:42.038 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-07-08T14:16:42.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:16:42.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:16:45.108 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:16:45.922 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~277s 2023-07-08T14:16:45.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~277s 2023-07-08T14:16:45.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~277s 2023-07-08T14:16:45.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~277s 2023-07-08T14:16:45.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~277s 2023-07-08T14:16:45.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~277s 2023-07-08T14:16:45.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~277s 2023-07-08T14:16:45.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~277s 2023-07-08T14:16:45.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~277s 2023-07-08T14:16:45.925 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~277s 2023-07-08T14:16:45.925 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~277s 2023-07-08T14:16:45.925 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~277s 2023-07-08T14:16:45.925 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~277s 2023-07-08T14:16:45.925 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~277s 2023-07-08T14:16:45.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~277s 2023-07-08T14:16:45.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~277s 2023-07-08T14:16:45.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~277s 2023-07-08T14:16:50.928 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~282s 2023-07-08T14:16:50.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~282s 2023-07-08T14:16:50.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~282s 2023-07-08T14:16:50.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~282s 2023-07-08T14:16:50.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~282s 2023-07-08T14:16:50.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~282s 2023-07-08T14:16:50.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~282s 2023-07-08T14:16:50.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~282s 2023-07-08T14:16:50.931 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~282s 2023-07-08T14:16:50.931 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~282s 2023-07-08T14:16:50.931 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~282s 2023-07-08T14:16:50.931 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~282s 2023-07-08T14:16:50.931 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~282s 2023-07-08T14:16:50.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~282s 2023-07-08T14:16:50.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~282s 2023-07-08T14:16:50.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~282s 2023-07-08T14:16:50.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~282s 2023-07-08T14:16:54.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:16:54.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:16:55.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~287s 2023-07-08T14:16:55.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~287s 2023-07-08T14:16:55.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~287s 2023-07-08T14:16:55.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~287s 2023-07-08T14:16:55.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~287s 2023-07-08T14:16:55.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~287s 2023-07-08T14:16:55.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~287s 2023-07-08T14:16:55.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~287s 2023-07-08T14:16:55.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~287s 2023-07-08T14:16:55.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~287s 2023-07-08T14:16:55.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~287s 2023-07-08T14:16:55.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~287s 2023-07-08T14:16:55.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~287s 2023-07-08T14:16:55.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~287s 2023-07-08T14:16:55.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~287s 2023-07-08T14:16:55.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~287s 2023-07-08T14:16:55.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~287s 2023-07-08T14:16:57.172 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:17:00.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~292s 2023-07-08T14:17:00.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~292s 2023-07-08T14:17:00.946 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~292s 2023-07-08T14:17:00.946 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~292s 2023-07-08T14:17:00.947 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~292s 2023-07-08T14:17:00.947 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~292s 2023-07-08T14:17:00.947 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~292s 2023-07-08T14:17:00.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~292s 2023-07-08T14:17:00.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~292s 2023-07-08T14:17:00.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~292s 2023-07-08T14:17:00.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~292s 2023-07-08T14:17:00.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~292s 2023-07-08T14:17:00.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~292s 2023-07-08T14:17:00.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~292s 2023-07-08T14:17:00.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~292s 2023-07-08T14:17:00.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~292s 2023-07-08T14:17:00.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~292s 2023-07-08T14:17:05.956 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~297s 2023-07-08T14:17:05.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~297s 2023-07-08T14:17:05.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~297s 2023-07-08T14:17:05.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~297s 2023-07-08T14:17:05.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~297s 2023-07-08T14:17:05.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~297s 2023-07-08T14:17:05.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~297s 2023-07-08T14:17:05.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~297s 2023-07-08T14:17:05.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~297s 2023-07-08T14:17:05.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~297s 2023-07-08T14:17:05.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~297s 2023-07-08T14:17:05.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~297s 2023-07-08T14:17:05.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~297s 2023-07-08T14:17:05.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~297s 2023-07-08T14:17:05.960 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~297s 2023-07-08T14:17:05.960 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~297s 2023-07-08T14:17:05.960 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~297s 2023-07-08T14:17:09.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:17:09.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:17:10.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~302s 2023-07-08T14:17:10.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~302s 2023-07-08T14:17:10.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~302s 2023-07-08T14:17:10.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~302s 2023-07-08T14:17:10.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~302s 2023-07-08T14:17:10.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~302s 2023-07-08T14:17:10.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~302s 2023-07-08T14:17:10.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~302s 2023-07-08T14:17:10.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~302s 2023-07-08T14:17:10.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~302s 2023-07-08T14:17:10.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~302s 2023-07-08T14:17:10.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~302s 2023-07-08T14:17:10.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~302s 2023-07-08T14:17:10.967 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~302s 2023-07-08T14:17:10.967 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~302s 2023-07-08T14:17:10.967 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~302s 2023-07-08T14:17:10.967 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~302s 2023-07-08T14:17:10.967 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-07-08T14:17:10.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:17:10.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:17:12.240 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:17:12.241 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:17:15.244 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:17:15.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:17:21.242 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:17:21.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:17:26.247 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:17:26.248 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:17:26.249 INFO:tasks.ceph:Archiving crash dumps... 2023-07-08T14:17:26.251 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-08_12:15:41-fs-wip-rishabh-mdscaps-2-testing-default-smithi/7329068/remote/smithi012/crash 2023-07-08T14:17:26.253 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:17:26.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:17:26.388 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:17:26.389 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:17:35.396 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:17:35.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:17:38.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:17:38.393 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:17:38.480 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:17:38.481 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:17:41.484 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:17:41.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:17:49.556 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:17:49.557 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:17:49.557 ERROR:tasks.daemonwatchdog.daemon_watchdog:ignoring exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/daemonwatchdog.py", line 61, in bark mount.umount_wait(force=True) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/kernel_mount.py", line 183, in umount_wait if not self.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:17:49.564 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:17:49.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:17:54.571 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:17:54.572 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:17:57.648 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:17:57.649 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:18:00.652 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:18:00.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:18:00.720 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:18:00.721 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:18:00.722 INFO:tasks.ceph:Shutting down logrotate 2023-07-08T14:18:00.723 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1940, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2744, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2576, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 2342, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 1957, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 256, in ceph_log logrotater.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 221, in end self.thread.get() File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/ceph.py", line 183, in invoke_logrotate procs = ctx.cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:18:00.743 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-08T14:18:00.756 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:18:00.770 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-08T14:18:00.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:18:00.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:18:03.824 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:18:06.652 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:18:06.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:18:16.660 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:18:16.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:18:21.664 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:18:22.800 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:18:33.676 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:18:33.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:18:34.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:18:34.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:18:39.809 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:18:39.824 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:18:42.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:18:42.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:18:47.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:18:47.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:18:52.822 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:18:52.823 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:18:52.823 ERROR:tasks.daemonwatchdog.daemon_watchdog:ignoring exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/daemonwatchdog.py", line 61, in bark mount.umount_wait(force=True) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/kernel_mount.py", line 183, in umount_wait if not self.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:18:52.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-07-08T14:18:52.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:18:53.972 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:18:53.973 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:18:54.030 ERROR:teuthology.run_tasks:Manager failed: install Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/install/util.py", line 148, in ship_utilities yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/install/__init__.py", line 626, in task yield File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/install/__init__.py", line 220, in install yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/install/util.py", line 150, in ship_utilities _remove_utilities(ctx, filenames) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/install/util.py", line 121, in _remove_utilities ctx.cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:18:54.045 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-08T14:18:54.056 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-08T14:18:54.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:18:54.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:18:57.040 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:19:06.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:19:06.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:19:09.104 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:19:11.344 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-08T14:19:20.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-07-08T14:19:20.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:19:21.106 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:19:21.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:19:26.113 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-08T14:19:29.173 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:19:32.177 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:19:32.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:19:38.116 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-07-08T14:19:38.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:19:43.122 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:19:43.123 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:19:43.124 ERROR:teuthology.run_tasks:Manager failed: clock Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/clock.py", line 66, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/clock.py", line 72, in task cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:19:43.135 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-08T14:19:43.147 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-08T14:19:43.148 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-08T14:19:43.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:19:43.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:19:47.753 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-08T14:19:50.756 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-07-08T14:19:50.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:19:53.498 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:19:56.565 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-08T14:19:56.565 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:19:56.566 ERROR:tasks.daemonwatchdog.daemon_watchdog:ignoring exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/daemonwatchdog.py", line 61, in bark mount.umount_wait(force=True) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/kernel_mount.py", line 183, in umount_wait if not self.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi160 2023-07-08T14:19:56.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-07-08T14:19:56.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:19:59.632 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-08T14:20:02.500 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:20:02.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:20:08.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-07-08T14:20:08.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:20:10.580 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:20:11.696 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-08T14:20:22.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:20:22.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:20:27.597 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-07-08T14:20:27.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:20:32.601 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:20:35.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:20:35.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:20:40.609 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-08T14:20:43.612 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-07-08T14:20:43.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:20:48.617 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:20:48.618 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:20:48.618 ERROR:teuthology.run_tasks:Manager failed: selinux Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/__init__.py", line 136, in __exit__ self.teardown() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/selinux.py", line 165, in teardown self.archive_log() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/selinux.py", line 195, in archive_log self.cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:20:48.628 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-08T14:20:48.645 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-08T14:20:48.668 INFO:teuthology.task.internal:Duration was 6887.133393 seconds 2023-07-08T14:20:48.669 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-08T14:20:48.703 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-08T14:20:48.703 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:20:48.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:20:53.713 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-07-08T14:20:53.713 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:20:53.713 ERROR:tasks.daemonwatchdog.daemon_watchdog:ignoring exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/daemonwatchdog.py", line 61, in bark mount.umount_wait(force=True) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/kernel_mount.py", line 183, in umount_wait if not self.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi160 2023-07-08T14:20:56.784 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:21:05.792 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:21:05.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:21:08.848 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:21:20.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:21:20.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:21:23.924 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:21:26.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:21:26.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:21:26.992 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:21:26.993 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:21:26.994 ERROR:teuthology.run_tasks:Manager failed: internal.syslog Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/internal/syslog.py", line 76, in syslog yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/internal/syslog.py", line 85, in syslog cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:21:27.007 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-08T14:21:27.020 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-08T14:21:27.020 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:21:27.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:21:30.096 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:21:39.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:21:39.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:21:42.160 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:21:54.172 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:21:54.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:21:57.236 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:22:00.240 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:22:00.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:22:00.304 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:22:00.306 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:22:00.307 ERROR:teuthology.run_tasks:Manager failed: internal.sudo Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/internal/__init__.py", line 452, in sudo yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/internal/__init__.py", line 455, in sudo ctx.cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:22:00.319 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-08T14:22:00.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:22:00.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:22:03.408 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:22:12.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:22:12.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:22:15.472 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:22:27.484 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:22:27.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:22:30.544 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:22:33.548 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:22:33.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:22:33.616 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:22:33.617 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:22:33.618 ERROR:teuthology.run_tasks:Manager failed: internal.coredump Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/internal/__init__.py", line 488, in coredump yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/internal/__init__.py", line 492, in coredump cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:22:33.628 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-08T14:22:33.640 INFO:teuthology.task.internal:Transferring archived files... 2023-07-08T14:22:33.641 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-08_12:15:41-fs-wip-rishabh-mdscaps-2-testing-default-smithi/7329068/remote/smithi012 2023-07-08T14:22:33.641 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:22:33.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:22:36.720 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:22:45.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:22:45.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:22:48.784 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:23:00.797 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:23:00.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:23:03.856 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:23:06.860 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:23:06.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:23:06.928 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:23:06.929 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:23:06.930 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/internal/__init__.py", line 392, in archive yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/internal/__init__.py", line 418, in archive misc.pull_directory(rem, archive_dir, path, maybe_compress) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/misc.py", line 748, in pull_directory r = remote.get_tar_stream(remotedir, sudo=True, compress=False) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 661, in get_tar_stream return self.run(args=args, wait=False, stdout=run.PIPE) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:23:06.940 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-08T14:23:06.953 INFO:teuthology.task.internal:Not uploading archives. 2023-07-08T14:23:06.953 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-08T14:23:06.977 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-08T14:23:06.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:23:06.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:23:10.032 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:23:19.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:23:19.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:23:22.100 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:23:34.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:23:34.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:23:37.168 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:23:40.172 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-07-08T14:23:40.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T14:23:40.240 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-08T14:23:40.241 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-08T14:23:40.241 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 139, in task forced = umount_all() File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/kclient.py", line 119, in umount_all if mount.is_mounted(): File "/home/teuthworker/src/git.ceph.com_ceph-c_f3a06ccd0ae55e9794bc8ce562926656b2738063/qa/tasks/cephfs/mount.py", line 220, in is_mounted file = self.client_remote.read_file('/proc/self/mounts',stdout=StringIO()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 253, in read_file proc = self.run(args=args, stdout=stdout, stderr=StringIO(), File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/task/internal/__init__.py", line 54, in base ctx.cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi012 2023-07-08T14:23:40.251 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-08T14:23:40.271 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-08T14:23:40.295 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: Exception: Cannot connect to remote host smithi012 2023-07-08T14:23:40.296 INFO:teuthology.nuke:Checking targets against current locks 2023-07-08T14:23:40.335 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run.py", line 411, in main run_tasks(tasks=config['tasks'], ctx=fake_ctx) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 220, in run_tasks raise SystemExit(1) SystemExit: 1 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/virtualenv/bin/teuthology", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/scripts/run.py", line 38, in main teuthology.run.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run.py", line 414, in main report_outcome(config, archive, fake_ctx.summary, fake_ctx) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run.py", line 271, in report_outcome nuke(fake_ctx, fake_ctx.lock) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/nuke/__init__.py", line 248, in nuke if ctx.name and ctx.name not in status.get('description', ""): TypeError: argument of type 'NoneType' is not iterable